Sep 15 11:23:08.196438 ip-10-0-138-153 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 15 11:23:08.196450 ip-10-0-138-153 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 15 11:23:08.196457 ip-10-0-138-153 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 15 11:23:08.196598 ip-10-0-138-153 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 15 11:23:19.459138 ip-10-0-138-153 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 15 11:23:19.459161 ip-10-0-138-153 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 526137f60fd240d39dec47d5fb6d511a --
Sep 15 11:24:57.353718 ip-10-0-138-153 systemd[1]: Starting Kubernetes Kubelet...
Sep 15 11:24:57.785907 ip-10-0-138-153 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 11:24:57.785907 ip-10-0-138-153 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 15 11:24:57.785907 ip-10-0-138-153 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 11:24:57.785907 ip-10-0-138-153 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 15 11:24:57.785907 ip-10-0-138-153 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 11:24:57.786892 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.786808    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 15 11:24:57.790084 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790070    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 11:24:57.790084 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790084    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790090    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790093    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790097    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790102    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790105    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790108    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790111    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790113    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790116    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790119    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790126    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790129    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790132    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790134    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790137    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790139    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790142    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790144    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 11:24:57.790152 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790147    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790149    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790154    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790157    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790161    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790164    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790166    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790169    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790172    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790175    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790177    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790180    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790183    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790185    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790188    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790190    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790193    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790196    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790200    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 11:24:57.790702 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790203    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790206    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790208    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790211    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790214    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790262    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790266    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790269    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790272    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790275    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790278    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790282    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790284    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790287    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790289    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790292    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790295    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790298    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790301    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790303    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 11:24:57.791165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790306    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790309    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790312    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790314    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790317    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790319    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790322    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790325    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790327    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790330    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790332    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790334    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790338    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790341    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790343    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790345    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790348    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790350    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790353    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790356    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 11:24:57.791721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790359    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790361    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790364    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790367    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790370    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790372    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790375    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790377    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790380    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790383    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790385    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790388    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790390    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790393    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790395    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790397    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790400    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790403    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790405    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 11:24:57.792208 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.790407    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791138    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791146    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791150    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791152    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791155    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791158    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791161    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791164    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791169    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791173    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791176    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791179    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791181    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791184    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791187    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791190    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791193    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791195    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 11:24:57.792691 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791199    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791201    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791204    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791206    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791209    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791212    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791214    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791217    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791219    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791222    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791224    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791227    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791231    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791234    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791237    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791240    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791244    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791246    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791249    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791252    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 11:24:57.793165 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791256    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791259    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791262    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791267    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791270    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791273    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791275    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791278    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791281    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791284    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791286    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791289    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791292    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791294    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791296    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791299    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791302    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791304    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791307    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791310    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 11:24:57.793684 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791313    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791315    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791318    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791321    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791323    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791325    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791328    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791331    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791333    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791336    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791338    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791341    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791344    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791347    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791349    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791352    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791355    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791357    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791360    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791362    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 11:24:57.794220 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791365    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791367    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791370    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791372    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791375    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791377    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791379    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791382    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791384    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791387    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791389    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791392    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791394    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791397    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791400    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791402    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791405    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791407    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791410    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 11:24:57.794733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791412    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791415    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791803    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791809    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791813    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791815    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791819    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791821    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791824    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791827    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791829    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791832    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791834    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791837    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791839    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791843    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791846    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791849    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791852    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791854    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 11:24:57.795197 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791857    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791859    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791862    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791864    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791867    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791870    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791873    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791875    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791878    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791880    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791883    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791885    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791888    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791890    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791895    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791897    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791900    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791902    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791905    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791908    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 11:24:57.795712 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791910    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791913    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791916    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791918    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791921    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791924    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791928    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791931    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791933    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791936    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791940    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791942    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791945    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791947    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791950    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791954    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791957    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791962    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791965    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 11:24:57.796207 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791968    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791971    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791973    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791976    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791978    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791981    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791983    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791987    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791990    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791992    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791995    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.791997    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792000    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792004    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792006    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792009    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792012    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792014    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792017    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792020    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 11:24:57.796696 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792022    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792025    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792027    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792030    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792032    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792035    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792037    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792040    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792042    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792045    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792049    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792052    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792054    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792057    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792059    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792062    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792064    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792067    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792069    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792074    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 11:24:57.797174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792076    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792079    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792145    2560 flags.go:64] FLAG: --address="0.0.0.0"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792182    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792189    2560 flags.go:64] FLAG: --anonymous-auth="true"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792193    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792200    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792203    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792207    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792211    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792214    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792217    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792221    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792224    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792227    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792230    2560 flags.go:64] FLAG: --cgroup-root=""
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792233    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792236    2560 flags.go:64] FLAG: --client-ca-file=""
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792239    2560 flags.go:64] FLAG: --cloud-provider="external"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792242    2560 flags.go:64] FLAG: --cluster-dns="[]"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792246    2560 flags.go:64] FLAG: --cluster-domain=""
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792249    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792256    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792259    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 15 11:24:57.797701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792262    2560 flags.go:64] FLAG: --container-log-max-files="5"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792266    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792269    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792272    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792275    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792278    2560 flags.go:64] FLAG: --contention-profiling="false"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792281    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792285    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792288    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792291    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792295    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792298    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792301    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792304    2560 flags.go:64] FLAG: --enable-load-reader="false"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792307    2560 flags.go:64] FLAG: --enable-server="true"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792310    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792315    2560 flags.go:64] FLAG: --event-burst="100"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792319    2560 flags.go:64] FLAG: --event-qps="50"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792321    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792324    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792327    2560 flags.go:64] FLAG: --eviction-hard=""
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792331    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792334    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792337    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792340    2560 flags.go:64] FLAG: --eviction-soft=""
Sep 15 11:24:57.798278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792342    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792345    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792348    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792351    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792354    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792358    2560 flags.go:64] FLAG: --fail-swap-on="true"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792361    2560 flags.go:64] FLAG: --feature-gates=""
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792365    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792368    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792371    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792374    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792377    2560 flags.go:64] FLAG: --healthz-port="10248"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792380    2560 flags.go:64] FLAG: --help="false"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792383    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-138-153.ec2.internal"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792387    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792390    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792393    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792396    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792400    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792402    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792405    2560 flags.go:64] FLAG: --image-service-endpoint=""
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792408    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792411    2560 flags.go:64] FLAG: --kube-api-burst="100"
Sep 15 11:24:57.798908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792414    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792418    2560 flags.go:64] FLAG: --kube-api-qps="50"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792420    2560 flags.go:64] FLAG: --kube-reserved=""
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792423    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792426    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792443    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792446    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792449    2560 flags.go:64] FLAG: --lock-file=""
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792452    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792455    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792458    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792464    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792467    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792470    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792473    2560 flags.go:64] FLAG: --logging-format="text"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792476    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792480    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792483    2560 flags.go:64] FLAG: --manifest-url=""
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792486    2560 flags.go:64] FLAG: --manifest-url-header=""
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792490    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792492    2560 flags.go:64] FLAG: --max-open-files="1000000"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792496    2560 flags.go:64] FLAG: --max-pods="110"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792499    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792504    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792507    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 15 11:24:57.799600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792510    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792513    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792516    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792519    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792525    2560 flags.go:64] FLAG: --node-status-max-images="50"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792528    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792532    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792535    2560 flags.go:64] FLAG: --pod-cidr=""
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792538    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792543    2560 flags.go:64] FLAG: --pod-manifest-path=""
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792546    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792550    2560 flags.go:64] FLAG: --pods-per-core="0"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792552    2560 flags.go:64] FLAG: --port="10250"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792555    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792558    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e399696e66acbdeb"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792561    2560 flags.go:64] FLAG: --qos-reserved=""
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792564    2560 flags.go:64] FLAG: --read-only-port="10255"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792567    2560 flags.go:64] FLAG: --register-node="true"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792570    2560 flags.go:64] FLAG: --register-with-taints=""
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792574    2560 flags.go:64] FLAG: --registry-burst="10"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792576    2560 flags.go:64] FLAG: --registry-qps="5"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792581    2560 flags.go:64] FLAG: --reserved-cpus=""
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792584    2560 flags.go:64] FLAG: --reserved-memory=""
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792587    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792590    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 15 11:24:57.800232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792593    2560 flags.go:64] FLAG: --rotate-certificates="false"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792596    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792599    2560 flags.go:64] FLAG: --runonce="false"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792602    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792605    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792608    2560 flags.go:64] FLAG: --seccomp-default="false"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792613    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792616    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792619    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792622    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792625    2560 flags.go:64] FLAG: --storage-driver-password="root"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792628    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792631    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792634    2560 flags.go:64] FLAG: --storage-driver-user="root"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792637    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792640    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792643    2560 flags.go:64] FLAG: --system-cgroups=""
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792646    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792651    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792654    2560 flags.go:64] FLAG: --tls-cert-file=""
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792657    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792661    2560 flags.go:64] FLAG: --tls-min-version=""
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792664    2560 flags.go:64] FLAG: --tls-private-key-file=""
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792667    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792670    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 15 11:24:57.800869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792673    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792676    2560 flags.go:64] FLAG: --v="2"
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792679    2560 flags.go:64] FLAG: --version="false"
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792684    2560 flags.go:64] FLAG: --vmodule=""
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792688    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.792691    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792818    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792822    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792825    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792828    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792831    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792834    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792837    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792840    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792843    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792846    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792848    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792851    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792853    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792856    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 11:24:57.801495 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792858    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792861    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792863    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792866    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792869    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792871    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792874    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792877    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792879    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792882    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792884    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792887    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792890    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792894    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792897    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792901    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792904    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792906    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792909    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792911    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 11:24:57.802025 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792914    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792918    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792921    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792924    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792927    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792930    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792934    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792936    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792939    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792942    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792945    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792947    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792950    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792953    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792955    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792958    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792960    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792963    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792966    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 11:24:57.802657 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792968    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792971    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792975    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792978    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792981    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792985    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792988    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792990    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792994    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.792997    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793000    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793003    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793005    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793008    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793011    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793013    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793016    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793018    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793021    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 11:24:57.803400 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793024    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793026    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793029    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793031    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793034    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793036    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793039    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793041    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793044    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793046    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793049    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793051    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793054    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793057    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793060    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793062    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793065    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793067    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793070    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793072    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793075    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 11:24:57.803938 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793079    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 11:24:57.804482 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793081    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 11:24:57.804482 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793084    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 11:24:57.804482 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793086    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 11:24:57.804482 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793089    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 11:24:57.804482 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.793092    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 11:24:57.804482 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.793096    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 11:24:57.804482 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.800685    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 15 11:24:57.804482 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.800701    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 15 11:24:57.804482 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.801577    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 15 11:24:57.804482 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.801585    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 15 11:24:57.804482 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801648    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 11:24:57.804482 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801654    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 11:24:57.804482 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801658    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 11:24:57.804482 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801661    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 11:24:57.804482 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801664    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801667    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801670    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801673    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801676    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801680    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801685    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801688    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801691    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801694    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801697    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801700    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801703    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801706    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801708    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801711    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801713    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801716    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801719    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 11:24:57.804856 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801722    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801724    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801727    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801730    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801732    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801735    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801740    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801744    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801747    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801750    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801753    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801756    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801759    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801762    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801765    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801768    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801770    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801773    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801776    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801778    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 11:24:57.805316 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801781    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801784    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801786    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801789    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801791    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801794    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801796    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801799    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801801    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801804    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801807    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801809    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801812    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801814    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801817    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801819    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801822    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801825    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801828    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801831    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 11:24:57.805882 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801834    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801837    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801840    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801844    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801847    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801850    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801852    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801855    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801858    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801860    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801863    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801866    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801868    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801871    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801873    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801875    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801878    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801881    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801883    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 11:24:57.806358 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801886    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801888    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801891    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801893    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801895    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801898    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801900    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801903    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801905    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801908    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801912    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801915    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801918    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801921    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801923    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801926    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 11:24:57.806854 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.801929    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 11:24:57.807279 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.801934    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 11:24:57.807279 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802049    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 11:24:57.807279 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802054    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 11:24:57.807279 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802057    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 11:24:57.807279 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802061    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 11:24:57.807279 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802065    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 11:24:57.807279 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802068    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 11:24:57.807279 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802071    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 11:24:57.807279 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802073    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 11:24:57.807279 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802076    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 11:24:57.807279 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802079    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 11:24:57.807279 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802082    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 11:24:57.807279 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802086    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 11:24:57.807279 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802089    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802091    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802094    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802097    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802100    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802103    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802106    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802110    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802113    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802117    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802120    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802122    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802125    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802127    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802130    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802133    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802135    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802139    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802141    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 11:24:57.807642 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802144    2560 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802146    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802149    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802151    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802154    2560 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802156    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802159    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802161    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802164    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802166    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802169    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802171    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802174    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802176    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802179    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802181    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802184    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802186    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802189    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802191    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 11:24:57.808104 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802194    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802196    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802199    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802201    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802204    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802206    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802209    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802211    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802214    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802216    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802219    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802222    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802224    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802227    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802229    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802232    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802234    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802237    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802239    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802241    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 11:24:57.808671 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802244    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802246    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802249    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802251    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802254    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802256    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802258    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802261    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802264    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802266    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802269    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802271    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802274    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802276    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802279    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802281    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802284    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802286    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802289    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802291    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 11:24:57.809159 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802294    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 11:24:57.809683 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802296    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 11:24:57.809683 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802299    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 11:24:57.809683 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802336    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 11:24:57.809683 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802371    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 11:24:57.809683 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802376    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 11:24:57.809683 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802380    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 11:24:57.809683 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:24:57.802384    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 11:24:57.809683 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.802393    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 11:24:57.809683 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.802720    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 15 11:24:57.809683 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.805370    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 15 11:24:57.809683 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.806508    2560 server.go:1019] "Starting client certificate rotation"
Sep 15 11:24:57.809683 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.806601    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 11:24:57.809683 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.806638    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 11:24:57.810014 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.809299    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 11:24:57.810404 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.810387    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 11:24:57.821139 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.821121    2560 log.go:25] "Validated CRI v1 runtime API"
Sep 15 11:24:57.826551 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.826534    2560 log.go:25] "Validated CRI v1 image API"
Sep 15 11:24:57.827337 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.827321    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 15 11:24:57.833168 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.833151    2560 fs.go:135] Filesystem UUIDs: map[3ddbc372-4657-434b-b1c9-07bffedaffae:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a5cd4ce9-921c-4629-beb9-4067772a8be5:/dev/nvme0n1p3]
Sep 15 11:24:57.833220 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.833168    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 15 11:24:57.838894 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.838787    2560 manager.go:216] Machine: {Timestamp:2026-09-15 11:24:57.837359011 +0000 UTC m=+0.378263914 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098740 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fa1eddc834bc2ed0247fa034683dc SystemUUID:ec2fa1ed-dc83-4bc2-ed02-47fa034683dc BootID:526137f6-0fd2-40d3-9dec-47d5fb6d511a Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:cc:e6:f4:6d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:cc:e6:f4:6d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:12:2e:90:f9:49 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 15 11:24:57.838894 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.838890    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 15 11:24:57.839006 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.838965    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 15 11:24:57.840897 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.840877    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 11:24:57.843824 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.843632    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 15 11:24:57.843955 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.843829    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-153.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 15 11:24:57.844707 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.844697    2560 topology_manager.go:138] "Creating topology manager with none policy"
Sep 15 11:24:57.844746 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.844709    2560 container_manager_linux.go:309] "Creating device plugin manager"
Sep 15 11:24:57.844746 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.844717    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 11:24:57.845514 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.845504    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 11:24:57.846718 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.846708    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 15 11:24:57.852118 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.852106    2560 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 11:24:57.852230 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.852214    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 15 11:24:57.852263 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.852242    2560 kubelet.go:486] "Attempting to sync node with API server"
Sep 15 11:24:57.852263 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.852252    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 15 11:24:57.852318 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.852268    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 15 11:24:57.852318 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.852277    2560 kubelet.go:398] "Adding apiserver pod source"
Sep 15 11:24:57.852318 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.852285    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 15 11:24:57.852664 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.852653    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 11:24:57.852712 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.852671    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 11:24:57.855162 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.855136    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 15 11:24:57.857346 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.857330    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 15 11:24:57.857420 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.857368    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 15 11:24:57.860376 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.860361    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 15 11:24:57.860475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.860382    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 15 11:24:57.860475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.860393    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 15 11:24:57.860475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.860400    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 15 11:24:57.860475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.860412    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 15 11:24:57.860475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.860419    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 15 11:24:57.860475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.860452    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 15 11:24:57.860475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.860460    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 15 11:24:57.860475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.860470    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 15 11:24:57.860711 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.860479    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 15 11:24:57.860711 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.860492    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 15 11:24:57.860711 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.860500    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 15 11:24:57.861413 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.861403    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 15 11:24:57.861504 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.861416    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 15 11:24:57.864041 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:57.864019    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 15 11:24:57.864041 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.864030    2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-153.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 15 11:24:57.864228 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:57.864212    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-153.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 15 11:24:57.865221 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.865208    2560 server.go:1268] "Started kubelet"
Sep 15 11:24:57.865313 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.865282    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 15 11:24:57.865367 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.865302    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 15 11:24:57.865413 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.865369    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 15 11:24:57.866030 ip-10-0-138-153 systemd[1]: Started Kubernetes Kubelet.
Sep 15 11:24:57.867833 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.867558    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 15 11:24:57.867986 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.867961    2560 server.go:310] "Adding debug handlers to kubelet server"
Sep 15 11:24:57.872212 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:57.871831    2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 15 11:24:57.872740 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:57.871669    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-153.ec2.internal.18d57ad8a5f0b1e8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-153.ec2.internal,UID:ip-10-0-138-153.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-153.ec2.internal,},FirstTimestamp:2026-09-15 11:24:57.865179624 +0000 UTC m=+0.406084528,LastTimestamp:2026-09-15 11:24:57.865179624 +0000 UTC m=+0.406084528,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-153.ec2.internal,}"
Sep 15 11:24:57.876217 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.876199    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 15 11:24:57.876873 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.876860    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 15 11:24:57.877038 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:57.877015    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-153.ec2.internal\" not found"
Sep 15 11:24:57.877717 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.877699    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 15 11:24:57.877717 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.877699    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 15 11:24:57.877834 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.877716    2560 factory.go:55] Registering systemd factory
Sep 15 11:24:57.877834 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.877727    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 15 11:24:57.877834 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.877732    2560 factory.go:223] Registration of the systemd container factory successfully
Sep 15 11:24:57.877933 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.877836    2560 reconstruct.go:97] "Volume reconstruction finished"
Sep 15 11:24:57.877933 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.877850    2560 reconciler.go:29] "Reconciler: start to sync state"
Sep 15 11:24:57.877988 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.877971    2560 factory.go:153] Registering CRI-O factory
Sep 15 11:24:57.877988 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.877981    2560 factory.go:223] Registration of the crio container factory successfully
Sep 15 11:24:57.878083 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.878038    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 15 11:24:57.878083 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.878051    2560 factory.go:103] Registering Raw factory
Sep 15 11:24:57.878083 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.878071    2560 manager.go:1196] Started watching for new ooms in manager
Sep 15 11:24:57.879022 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:57.878994    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-153.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 15 11:24:57.879107 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.879051    2560 manager.go:349] Starting recovery of all containers
Sep 15 11:24:57.889806 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:57.889629    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 15 11:24:57.890039 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.890002    2560 manager.go:354] Recovery completed
Sep 15 11:24:57.895751 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.895739    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 11:24:57.897330 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.897313    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 11:24:57.897390 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.897345    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 11:24:57.897390 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.897359    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientPID"
Sep 15 11:24:57.897941 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.897921    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 15 11:24:57.897941 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.897938    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 11:24:57.898090 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.897941    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 15 11:24:57.898090 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.897997    2560 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 11:24:57.898659 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.898641    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 11:24:57.898717 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.898673    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 11:24:57.898717 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.898689    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientPID"
Sep 15 11:24:57.899132 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.899115    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gg99v"
Sep 15 11:24:57.900284 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.900270    2560 policy_none.go:49] "None policy: Start"
Sep 15 11:24:57.900357 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.900289    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 15 11:24:57.900357 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.900303    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 15 11:24:57.902275 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.902263    2560 policy_none.go:47] "Start"
Sep 15 11:24:57.908176 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.908121    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gg99v"
Sep 15 11:24:57.908276 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:57.908203    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-153.ec2.internal.18d57ad8a7db4283  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-153.ec2.internal,UID:ip-10-0-138-153.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-153.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-153.ec2.internal,},FirstTimestamp:2026-09-15 11:24:57.897329283 +0000 UTC m=+0.438234185,LastTimestamp:2026-09-15 11:24:57.897329283 +0000 UTC m=+0.438234185,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-153.ec2.internal,}"
Sep 15 11:24:57.909933 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:57.909855    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-153.ec2.internal.18d57ad8a7db99e8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-153.ec2.internal,UID:ip-10-0-138-153.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-153.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-153.ec2.internal,},FirstTimestamp:2026-09-15 11:24:57.897351656 +0000 UTC m=+0.438256559,LastTimestamp:2026-09-15 11:24:57.897351656 +0000 UTC m=+0.438256559,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-153.ec2.internal,}"
Sep 15 11:24:57.929051 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.929029    2560 manager.go:337] "Starting Device Plugin manager"
Sep 15 11:24:57.929142 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:57.929073    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 15 11:24:57.929142 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.929085    2560 server.go:87] "Starting device plugin registration server"
Sep 15 11:24:57.929378 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.929366    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 15 11:24:57.929450 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.929380    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 15 11:24:57.929522 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.929492    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 15 11:24:57.930001 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.929609    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 15 11:24:57.930001 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.929631    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 15 11:24:57.931687 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:57.931605    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 15 11:24:57.931687 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:57.931667    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-153.ec2.internal\" not found"
Sep 15 11:24:57.978405 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.978378    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 15 11:24:57.979494 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.979479    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 15 11:24:57.979565 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.979526    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 15 11:24:57.979565 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.979552    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 15 11:24:57.979653 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:57.979634    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 15 11:24:57.981881 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:57.981854    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 11:24:58.029959 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.029942    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 11:24:58.030771 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.030754    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 11:24:58.030847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.030782    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 11:24:58.030847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.030792    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientPID"
Sep 15 11:24:58.030847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.030835    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.040621 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.040581    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.040621 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:58.040601    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-153.ec2.internal\": node \"ip-10-0-138-153.ec2.internal\" not found"
Sep 15 11:24:58.070652 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:58.070632    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-153.ec2.internal\" not found"
Sep 15 11:24:58.079865 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.079844    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-153.ec2.internal"]
Sep 15 11:24:58.079945 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.079928    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 11:24:58.081192 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.081177    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 11:24:58.081272 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.081209    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 11:24:58.081272 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.081219    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientPID"
Sep 15 11:24:58.083463 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.083451    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 11:24:58.083585 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.083572    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.083630 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.083596    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 11:24:58.084130 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.084116    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 11:24:58.084201 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.084138    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 11:24:58.084201 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.084120    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 11:24:58.084201 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.084148    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientPID"
Sep 15 11:24:58.084201 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.084165    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 11:24:58.084201 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.084175    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientPID"
Sep 15 11:24:58.087093 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.087078    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.087160 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.087105    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 11:24:58.087777 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.087762    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 11:24:58.087850 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.087790    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 11:24:58.087850 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.087803    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientPID"
Sep 15 11:24:58.099979 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:58.099960    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-153.ec2.internal\" not found" node="ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.104470 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:58.104455    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-153.ec2.internal\" not found" node="ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.171018 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:58.170999    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-153.ec2.internal\" not found"
Sep 15 11:24:58.179134 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.179119    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f7ed78fa2944f8540ed193d79ae916e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal\" (UID: \"f7ed78fa2944f8540ed193d79ae916e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.179215 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.179141    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7ed78fa2944f8540ed193d79ae916e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal\" (UID: \"f7ed78fa2944f8540ed193d79ae916e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.179215 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.179164    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1142610769d3e76a0d4f81b8e01ff4c0-config\") pod \"kube-apiserver-proxy-ip-10-0-138-153.ec2.internal\" (UID: \"1142610769d3e76a0d4f81b8e01ff4c0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.271574 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:58.271553    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-153.ec2.internal\" not found"
Sep 15 11:24:58.279868 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.279840    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7ed78fa2944f8540ed193d79ae916e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal\" (UID: \"f7ed78fa2944f8540ed193d79ae916e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.279926 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.279875    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1142610769d3e76a0d4f81b8e01ff4c0-config\") pod \"kube-apiserver-proxy-ip-10-0-138-153.ec2.internal\" (UID: \"1142610769d3e76a0d4f81b8e01ff4c0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.279926 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.279892    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f7ed78fa2944f8540ed193d79ae916e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal\" (UID: \"f7ed78fa2944f8540ed193d79ae916e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.279988 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.279930    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1142610769d3e76a0d4f81b8e01ff4c0-config\") pod \"kube-apiserver-proxy-ip-10-0-138-153.ec2.internal\" (UID: \"1142610769d3e76a0d4f81b8e01ff4c0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.279988 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.279932    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f7ed78fa2944f8540ed193d79ae916e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal\" (UID: \"f7ed78fa2944f8540ed193d79ae916e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.279988 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.279974    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7ed78fa2944f8540ed193d79ae916e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal\" (UID: \"f7ed78fa2944f8540ed193d79ae916e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.372260 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:58.372215    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-153.ec2.internal\" not found"
Sep 15 11:24:58.402700 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.402677    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.407564 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:58.407487    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-153.ec2.internal"
Sep 15 11:24:58.473209 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:58.473186    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-153.ec2.internal\" not found"
Sep 15 11:24:59.621854 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:59.621832    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-153.ec2.internal\" not found"
Sep 15 11:24:59.722346 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:59.722290    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-153.ec2.internal\" not found"
Sep 15 11:24:59.822941 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:59.822917    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-153.ec2.internal\" not found"
Sep 15 11:24:59.854350 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:59.854328    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 15 11:24:59.854783 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:59.854461    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 11:24:59.923952 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:24:59.923920    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-153.ec2.internal\" not found"
Sep 15 11:24:59.965193 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:59.965148    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 11:19:58 +0000 UTC" deadline="2028-04-25 11:31:55.712705372 +0000 UTC"
Sep 15 11:24:59.965193 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:59.965174    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14112h6m55.747534296s"
Sep 15 11:24:59.980485 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:59.980433    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 11:24:59.981851 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:59.981836    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 11:24:59.981908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:59.981865    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 11:24:59.981908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:24:59.981877    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeHasSufficientPID"
Sep 15 11:25:00.014888 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:00.014863    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7ed78fa2944f8540ed193d79ae916e7.slice/crio-7e819ca1ead02af7d95f185427a6104f0d6e9088931d2452ab899e4dd2f34ba7 WatchSource:0}: Error finding container 7e819ca1ead02af7d95f185427a6104f0d6e9088931d2452ab899e4dd2f34ba7: Status 404 returned error can't find the container with id 7e819ca1ead02af7d95f185427a6104f0d6e9088931d2452ab899e4dd2f34ba7
Sep 15 11:25:00.015853 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:00.015818    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1142610769d3e76a0d4f81b8e01ff4c0.slice/crio-562b5fea384b93ae515161ed8bf98b6562680db4517dcf78ce2baa74514018f6 WatchSource:0}: Error finding container 562b5fea384b93ae515161ed8bf98b6562680db4517dcf78ce2baa74514018f6: Status 404 returned error can't find the container with id 562b5fea384b93ae515161ed8bf98b6562680db4517dcf78ce2baa74514018f6
Sep 15 11:25:00.023046 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.023032    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 11:25:00.024639 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:00.024608    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-153.ec2.internal\" not found"
Sep 15 11:25:00.031941 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.031529    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal" event={"ID":"f7ed78fa2944f8540ed193d79ae916e7","Type":"ContainerStarted","Data":"7e819ca1ead02af7d95f185427a6104f0d6e9088931d2452ab899e4dd2f34ba7"}
Sep 15 11:25:00.033252 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.033232    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-153.ec2.internal" event={"ID":"1142610769d3e76a0d4f81b8e01ff4c0","Type":"ContainerStarted","Data":"562b5fea384b93ae515161ed8bf98b6562680db4517dcf78ce2baa74514018f6"}
Sep 15 11:25:00.125053 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:00.125020    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-153.ec2.internal\" not found"
Sep 15 11:25:00.202854 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.202828    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 11:25:00.225362 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.225345    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal"
Sep 15 11:25:00.237820 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.237770    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 11:25:00.239425 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.239412    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-153.ec2.internal"
Sep 15 11:25:00.244943 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.244931    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 11:25:00.303365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.303344    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 11:25:00.497729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.497668    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 11:25:00.901575 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.901492    2560 apiserver.go:52] "Watching apiserver"
Sep 15 11:25:00.907112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.907082    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 15 11:25:00.908416 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.908388    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-153.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs","openshift-cluster-node-tuning-operator/tuned-zv9mn","openshift-multus/multus-additional-cni-plugins-lhdb5","openshift-multus/multus-dpznr","openshift-multus/network-metrics-daemon-p5wb6","openshift-network-diagnostics/network-check-target-sfjzw","openshift-network-operator/iptables-alerter-vb9xp","kube-system/global-pull-secret-syncer-4jcwq","kube-system/konnectivity-agent-5sb4n","openshift-dns/node-resolver-hq8r8","openshift-image-registry/node-ca-xfzbq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-mtm94"]
Sep 15 11:25:00.910926 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.910905    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:00.911123 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:00.911089    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:00.913031 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.912988    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hq8r8"
Sep 15 11:25:00.915257 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.915237    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qbmqv\""
Sep 15 11:25:00.915463 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.915446    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 15 11:25:00.915703 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.915683    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 15 11:25:00.916162 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.916148    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:00.917963 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.917860    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8xqtk\""
Sep 15 11:25:00.918074 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.918010    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 15 11:25:00.918155 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.918133    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 15 11:25:00.919988 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.918882    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:00.920659 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.920638    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 15 11:25:00.920986 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.920969    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.921068 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.921001    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ss95z\""
Sep 15 11:25:00.921068 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.921063    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 15 11:25:00.921408 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.921390    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 15 11:25:00.921599 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.921583    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 15 11:25:00.922196 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.921765    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 15 11:25:00.923697 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.922748    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 15 11:25:00.923697 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.923135    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-47zqv\""
Sep 15 11:25:00.923697 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.923403    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:00.923697 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:00.923457    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:00.925791 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.925422    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:00.925791 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.925487    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 15 11:25:00.925791 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:00.925489    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:00.928467 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.928040    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.930082 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.930055    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 15 11:25:00.930469 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.930448    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 15 11:25:00.930556 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.930494    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 15 11:25:00.930682 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.930628    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 15 11:25:00.930682 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.930667    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 15 11:25:00.930909 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.930831    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-m958l\""
Sep 15 11:25:00.930966 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.930912    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:00.931017 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.931000    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 15 11:25:00.933672 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.933434    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 15 11:25:00.933784 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.933699    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-77gk9\""
Sep 15 11:25:00.933784 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.933728    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 15 11:25:00.933888 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.933875    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 15 11:25:00.936324 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.936309    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5sb4n"
Sep 15 11:25:00.936796 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.936772    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-var-lib-cni-bin\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.936888 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.936812    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-sysctl-d\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:00.936888 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.936835    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-run\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:00.936888 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.936857    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1bd5c541-ba2b-4ab4-a874-9c0436910724-cnibin\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:00.936888 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.936880    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1bd5c541-ba2b-4ab4-a874-9c0436910724-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:00.937082 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.936902    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-dbus\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:00.937082 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.936925    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqtkn\" (UniqueName: \"kubernetes.io/projected/1bd5c541-ba2b-4ab4-a874-9c0436910724-kube-api-access-xqtkn\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:00.937082 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.936947    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbkd4\" (UniqueName: \"kubernetes.io/projected/83354a1c-ecad-424f-8efa-e6bf2e044fae-kube-api-access-fbkd4\") pod \"network-metrics-daemon-p5wb6\" (UID: \"83354a1c-ecad-424f-8efa-e6bf2e044fae\") " pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:00.937082 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.936970    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-systemd-units\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.937082 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.936994    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.937082 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937017    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/43830c40-9ac0-4143-b673-a73a6273075c-hosts-file\") pod \"node-resolver-hq8r8\" (UID: \"43830c40-9ac0-4143-b673-a73a6273075c\") " pod="openshift-dns/node-resolver-hq8r8"
Sep 15 11:25:00.937082 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937045    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-systemd\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:00.937082 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937075    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1bd5c541-ba2b-4ab4-a874-9c0436910724-system-cni-dir\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:00.937436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937098    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1bd5c541-ba2b-4ab4-a874-9c0436910724-cni-binary-copy\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:00.937436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937121    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-var-lib-kubelet\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.937436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937142    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-hostroot\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.937436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937165    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-multus-conf-dir\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.937436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937188    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-run-systemd\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.937436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937248    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbfhw\" (UniqueName: \"kubernetes.io/projected/43830c40-9ac0-4143-b673-a73a6273075c-kube-api-access-tbfhw\") pod \"node-resolver-hq8r8\" (UID: \"43830c40-9ac0-4143-b673-a73a6273075c\") " pod="openshift-dns/node-resolver-hq8r8"
Sep 15 11:25:00.937436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937278    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-sysconfig\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:00.937436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937302    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-slash\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.937436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937331    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-run-ovn\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.937436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937363    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-cnibin\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.937436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937389    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-run-k8s-cni-cncf-io\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.937436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937422    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs\") pod \"network-metrics-daemon-p5wb6\" (UID: \"83354a1c-ecad-424f-8efa-e6bf2e044fae\") " pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:00.937992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937448    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-etc-openvswitch\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.937992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937476    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-node-log\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.937992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937489    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-run-ovn-kubernetes\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.937992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937502    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-cni-bin\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.937992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937522    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c7a095f9-e6d1-40e1-be22-01493dc8f629-ovnkube-script-lib\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.937992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937638    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-modprobe-d\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:00.937992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937668    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-tmp\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:00.937992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937691    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1bd5c541-ba2b-4ab4-a874-9c0436910724-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:00.937992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937716    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/652d024a-faff-4a93-bedb-444cc57170b9-cni-binary-copy\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.937992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937792    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-var-lib-cni-multus\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.937992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937830    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-run-netns\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.937992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937860    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-run-openvswitch\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.937992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937910    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-lib-modules\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:00.937992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937946    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-tuned\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:00.937992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.937974    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1bd5c541-ba2b-4ab4-a874-9c0436910724-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938002    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-multus-socket-dir-parent\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938026    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-sysctl-conf\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938050    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-var-lib-kubelet\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938072    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-run-netns\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938096    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c7a095f9-e6d1-40e1-be22-01493dc8f629-ovn-node-metrics-cert\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938165    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-host\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938197    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-cni-netd\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938223    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-kubelet-config\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938246    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c7a095f9-e6d1-40e1-be22-01493dc8f629-env-overrides\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938282    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-var-lib-openvswitch\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938303    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-system-cni-dir\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938325    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-multus-cni-dir\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938346    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-etc-kubernetes\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938369    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938404    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkqsb\" (UniqueName: \"kubernetes.io/projected/c7a095f9-e6d1-40e1-be22-01493dc8f629-kube-api-access-lkqsb\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.938725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938461    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwnxg\" (UniqueName: \"kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg\") pod \"network-check-target-sfjzw\" (UID: \"9674a018-9086-45a1-b5cf-ef8e9dcd0366\") " pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:00.939459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938489    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43830c40-9ac0-4143-b673-a73a6273075c-tmp-dir\") pod \"node-resolver-hq8r8\" (UID: \"43830c40-9ac0-4143-b673-a73a6273075c\") " pod="openshift-dns/node-resolver-hq8r8"
Sep 15 11:25:00.939459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938512    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-kubernetes\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:00.939459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938535    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-run-multus-certs\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.939459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938556    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cptxg\" (UniqueName: \"kubernetes.io/projected/652d024a-faff-4a93-bedb-444cc57170b9-kube-api-access-cptxg\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.939459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938600    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-log-socket\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.939459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938620    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c7a095f9-e6d1-40e1-be22-01493dc8f629-ovnkube-config\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.939459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938643    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/652d024a-faff-4a93-bedb-444cc57170b9-multus-daemon-config\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:00.939459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938696    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-sys\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:00.939459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938720    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8k8b\" (UniqueName: \"kubernetes.io/projected/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-kube-api-access-k8k8b\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:00.939459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938740    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-kubelet\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:00.939459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938776    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 15 11:25:00.939459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.938980    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sv5sv\""
Sep 15 11:25:00.939459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.939012    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 15 11:25:00.939459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.939100    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vb9xp"
Sep 15 11:25:00.940230 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.940185    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 11:25:00.941114 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.941036    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 15 11:25:00.941254 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.941238    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 15 11:25:00.941447 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.941412    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 15 11:25:00.941513 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.941464    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xfzbq"
Sep 15 11:25:00.941672 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.941602    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jhvzr\""
Sep 15 11:25:00.943620 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.943602    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 15 11:25:00.943826 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.943810    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 15 11:25:00.943923 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.943827    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 15 11:25:00.944114 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.944096    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fl5hm\""
Sep 15 11:25:00.959875 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.959852    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9xgfz"
Sep 15 11:25:00.967531 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:00.967509    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9xgfz"
Sep 15 11:25:01.027093 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.027069    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 15 11:25:01.038978 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.038952    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-sys\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.039091 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.038983    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8k8b\" (UniqueName: \"kubernetes.io/projected/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-kube-api-access-k8k8b\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.039091 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.038999    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-kubelet\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.039091 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039020    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25l6v\" (UniqueName: \"kubernetes.io/projected/8b7525af-768b-49bb-9253-fb405d2f393a-kube-api-access-25l6v\") pod \"iptables-alerter-vb9xp\" (UID: \"8b7525af-768b-49bb-9253-fb405d2f393a\") " pod="openshift-network-operator/iptables-alerter-vb9xp"
Sep 15 11:25:01.039267 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039093    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-sys\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.039267 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039093    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-kubelet\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.039267 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039254    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmjjl\" (UniqueName: \"kubernetes.io/projected/dc0a2e63-3309-454b-890d-bc634e4ee43a-kube-api-access-cmjjl\") pod \"node-ca-xfzbq\" (UID: \"dc0a2e63-3309-454b-890d-bc634e4ee43a\") " pod="openshift-image-registry/node-ca-xfzbq"
Sep 15 11:25:01.039425 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039296    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-var-lib-cni-bin\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.039425 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039322    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dc0a2e63-3309-454b-890d-bc634e4ee43a-serviceca\") pod \"node-ca-xfzbq\" (UID: \"dc0a2e63-3309-454b-890d-bc634e4ee43a\") " pod="openshift-image-registry/node-ca-xfzbq"
Sep 15 11:25:01.039425 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039349    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-sysctl-d\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.039425 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039354    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-var-lib-cni-bin\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.039425 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039372    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-run\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.039425 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039408    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1bd5c541-ba2b-4ab4-a874-9c0436910724-cnibin\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:01.039425 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039424    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-run\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.039788 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039436    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1bd5c541-ba2b-4ab4-a874-9c0436910724-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:01.039788 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039459    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1bd5c541-ba2b-4ab4-a874-9c0436910724-cnibin\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:01.039788 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039461    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-dbus\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:01.039788 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039482    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-sysctl-d\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.039788 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039487    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b7525af-768b-49bb-9253-fb405d2f393a-host-slash\") pod \"iptables-alerter-vb9xp\" (UID: \"8b7525af-768b-49bb-9253-fb405d2f393a\") " pod="openshift-network-operator/iptables-alerter-vb9xp"
Sep 15 11:25:01.039788 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039511    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqtkn\" (UniqueName: \"kubernetes.io/projected/1bd5c541-ba2b-4ab4-a874-9c0436910724-kube-api-access-xqtkn\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:01.039788 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039535    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbkd4\" (UniqueName: \"kubernetes.io/projected/83354a1c-ecad-424f-8efa-e6bf2e044fae-kube-api-access-fbkd4\") pod \"network-metrics-daemon-p5wb6\" (UID: \"83354a1c-ecad-424f-8efa-e6bf2e044fae\") " pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:01.039788 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039548    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-dbus\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:01.039788 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039625    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1bd5c541-ba2b-4ab4-a874-9c0436910724-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:01.039788 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039556    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-systemd-units\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039792    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039748    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-systemd-units\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039831    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68swh\" (UniqueName: \"kubernetes.io/projected/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-kube-api-access-68swh\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039857    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/43830c40-9ac0-4143-b673-a73a6273075c-hosts-file\") pod \"node-resolver-hq8r8\" (UID: \"43830c40-9ac0-4143-b673-a73a6273075c\") " pod="openshift-dns/node-resolver-hq8r8"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039863    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039886    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-systemd\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039910    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1bd5c541-ba2b-4ab4-a874-9c0436910724-system-cni-dir\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039930    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1bd5c541-ba2b-4ab4-a874-9c0436910724-cni-binary-copy\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039932    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/43830c40-9ac0-4143-b673-a73a6273075c-hosts-file\") pod \"node-resolver-hq8r8\" (UID: \"43830c40-9ac0-4143-b673-a73a6273075c\") " pod="openshift-dns/node-resolver-hq8r8"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039958    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-var-lib-kubelet\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.039980    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-hostroot\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040001    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-multus-conf-dir\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040017    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-systemd\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040022    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-run-systemd\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040045    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbfhw\" (UniqueName: \"kubernetes.io/projected/43830c40-9ac0-4143-b673-a73a6273075c-kube-api-access-tbfhw\") pod \"node-resolver-hq8r8\" (UID: \"43830c40-9ac0-4143-b673-a73a6273075c\") " pod="openshift-dns/node-resolver-hq8r8"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040067    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-sysconfig\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.040214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040091    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-slash\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040114    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-run-ovn\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040136    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-etc-selinux\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040158    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-cnibin\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040178    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-var-lib-kubelet\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040180    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-run-k8s-cni-cncf-io\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040220    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs\") pod \"network-metrics-daemon-p5wb6\" (UID: \"83354a1c-ecad-424f-8efa-e6bf2e044fae\") " pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040246    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-etc-openvswitch\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040263    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-multus-conf-dir\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040281    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-node-log\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040305    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-run-ovn-kubernetes\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040312    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-run-systemd\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040329    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-cni-bin\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040353    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c7a095f9-e6d1-40e1-be22-01493dc8f629-ovnkube-script-lib\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040386    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-modprobe-d\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040407    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-tmp\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040433    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1bd5c541-ba2b-4ab4-a874-9c0436910724-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:01.040913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040458    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/652d024a-faff-4a93-bedb-444cc57170b9-cni-binary-copy\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040481    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-var-lib-cni-multus\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040506    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-run-netns\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040529    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-run-openvswitch\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040551    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-sysconfig\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040556    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040067    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1bd5c541-ba2b-4ab4-a874-9c0436910724-system-cni-dir\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040616    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1bd5c541-ba2b-4ab4-a874-9c0436910724-cni-binary-copy\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040215    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-run-k8s-cni-cncf-io\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040618    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-lib-modules\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040673    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-hostroot\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040744    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-slash\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040750    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-run-ovn\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.040832    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040858    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-modprobe-d\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040894    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-var-lib-cni-multus\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.040912    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs podName:83354a1c-ecad-424f-8efa-e6bf2e044fae nodeName:}" failed. No retries permitted until 2026-09-15 11:25:01.540893285 +0000 UTC m=+3.033670417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs") pod "network-metrics-daemon-p5wb6" (UID: "83354a1c-ecad-424f-8efa-e6bf2e044fae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 11:25:01.041662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040927    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-run-netns\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.040958    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-run-openvswitch\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041015    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-node-log\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041060    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-tuned\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041089    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1bd5c541-ba2b-4ab4-a874-9c0436910724-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041145    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-multus-socket-dir-parent\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041167    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-sysctl-conf\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041188    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-var-lib-kubelet\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041237    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-run-netns\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041243    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-lib-modules\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041263    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c7a095f9-e6d1-40e1-be22-01493dc8f629-ovn-node-metrics-cert\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041277    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-run-ovn-kubernetes\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041291    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/02e080c4-4b79-42c3-9ad1-319b5ca2df4c-agent-certs\") pod \"konnectivity-agent-5sb4n\" (UID: \"02e080c4-4b79-42c3-9ad1-319b5ca2df4c\") " pod="kube-system/konnectivity-agent-5sb4n"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041311    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-cni-bin\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041316    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-host\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041339    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-cni-netd\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041364    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-sys-fs\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.042533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041389    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-kubelet-config\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041473    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c7a095f9-e6d1-40e1-be22-01493dc8f629-env-overrides\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041498    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-socket-dir\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041507    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041522    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-registration-dir\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041578    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-var-lib-openvswitch\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041609    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-system-cni-dir\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041633    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-multus-cni-dir\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041655    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-etc-kubernetes\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041680    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041705    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkqsb\" (UniqueName: \"kubernetes.io/projected/c7a095f9-e6d1-40e1-be22-01493dc8f629-kube-api-access-lkqsb\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041733    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-device-dir\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041766    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwnxg\" (UniqueName: \"kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg\") pod \"network-check-target-sfjzw\" (UID: \"9674a018-9086-45a1-b5cf-ef8e9dcd0366\") " pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041799    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43830c40-9ac0-4143-b673-a73a6273075c-tmp-dir\") pod \"node-resolver-hq8r8\" (UID: \"43830c40-9ac0-4143-b673-a73a6273075c\") " pod="openshift-dns/node-resolver-hq8r8"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041823    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-kubernetes\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041829    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/652d024a-faff-4a93-bedb-444cc57170b9-cni-binary-copy\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041847    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-run-multus-certs\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041841    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c7a095f9-e6d1-40e1-be22-01493dc8f629-ovnkube-script-lib\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.043365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041894    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-cnibin\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041936    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-run-multus-certs\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041942    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cptxg\" (UniqueName: \"kubernetes.io/projected/652d024a-faff-4a93-bedb-444cc57170b9-kube-api-access-cptxg\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041966    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-kubernetes\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041975    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-log-socket\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042002    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c7a095f9-e6d1-40e1-be22-01493dc8f629-ovnkube-config\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042003    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-host-cni-netd\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042023    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-multus-cni-dir\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042032    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8b7525af-768b-49bb-9253-fb405d2f393a-iptables-alerter-script\") pod \"iptables-alerter-vb9xp\" (UID: \"8b7525af-768b-49bb-9253-fb405d2f393a\") " pod="openshift-network-operator/iptables-alerter-vb9xp"
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042055    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-etc-kubernetes\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042062    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/652d024a-faff-4a93-bedb-444cc57170b9-multus-daemon-config\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.042088    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042089    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/02e080c4-4b79-42c3-9ad1-319b5ca2df4c-konnectivity-ca\") pod \"konnectivity-agent-5sb4n\" (UID: \"02e080c4-4b79-42c3-9ad1-319b5ca2df4c\") " pod="kube-system/konnectivity-agent-5sb4n"
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042114    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc0a2e63-3309-454b-890d-bc634e4ee43a-host\") pod \"node-ca-xfzbq\" (UID: \"dc0a2e63-3309-454b-890d-bc634e4ee43a\") " pod="openshift-image-registry/node-ca-xfzbq"
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.042140    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret podName:7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b nodeName:}" failed. No retries permitted until 2026-09-15 11:25:01.542124631 +0000 UTC m=+3.034901771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret") pod "global-pull-secret-syncer-4jcwq" (UID: "7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b") : object "kube-system"/"original-pull-secret" not registered
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042173    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-var-lib-openvswitch\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.041976    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-host\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.044285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042513    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c7a095f9-e6d1-40e1-be22-01493dc8f629-env-overrides\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.045111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042531    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-multus-socket-dir-parent\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.045111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042590    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-var-lib-kubelet\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.045111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042593    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-etc-openvswitch\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.045111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042643    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-system-cni-dir\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.045111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042678    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/652d024a-faff-4a93-bedb-444cc57170b9-host-run-netns\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.045111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042698    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43830c40-9ac0-4143-b673-a73a6273075c-tmp-dir\") pod \"node-resolver-hq8r8\" (UID: \"43830c40-9ac0-4143-b673-a73a6273075c\") " pod="openshift-dns/node-resolver-hq8r8"
Sep 15 11:25:01.045111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042718    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-kubelet-config\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:01.045111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.042813    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-sysctl-conf\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.045111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.043035    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1bd5c541-ba2b-4ab4-a874-9c0436910724-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:01.045111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.043252    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c7a095f9-e6d1-40e1-be22-01493dc8f629-ovnkube-config\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.045111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.043445    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1bd5c541-ba2b-4ab4-a874-9c0436910724-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:01.045111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.043672    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/652d024a-faff-4a93-bedb-444cc57170b9-multus-daemon-config\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.045111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.043688    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c7a095f9-e6d1-40e1-be22-01493dc8f629-log-socket\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.045766 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.045502    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-etc-tuned\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.045766 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.045706    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-tmp\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.046354 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.046326    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c7a095f9-e6d1-40e1-be22-01493dc8f629-ovn-node-metrics-cert\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.047760 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.047738    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 11:25:01.047840 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.047765    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 11:25:01.047840 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.047779    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xwnxg for pod openshift-network-diagnostics/network-check-target-sfjzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 11:25:01.047840 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.047838    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg podName:9674a018-9086-45a1-b5cf-ef8e9dcd0366 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:01.547822743 +0000 UTC m=+3.040599871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xwnxg" (UniqueName: "kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg") pod "network-check-target-sfjzw" (UID: "9674a018-9086-45a1-b5cf-ef8e9dcd0366") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 11:25:01.048160 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.048137    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqtkn\" (UniqueName: \"kubernetes.io/projected/1bd5c541-ba2b-4ab4-a874-9c0436910724-kube-api-access-xqtkn\") pod \"multus-additional-cni-plugins-lhdb5\" (UID: \"1bd5c541-ba2b-4ab4-a874-9c0436910724\") " pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:01.048829 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.048808    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbfhw\" (UniqueName: \"kubernetes.io/projected/43830c40-9ac0-4143-b673-a73a6273075c-kube-api-access-tbfhw\") pod \"node-resolver-hq8r8\" (UID: \"43830c40-9ac0-4143-b673-a73a6273075c\") " pod="openshift-dns/node-resolver-hq8r8"
Sep 15 11:25:01.050249 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.050225    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbkd4\" (UniqueName: \"kubernetes.io/projected/83354a1c-ecad-424f-8efa-e6bf2e044fae-kube-api-access-fbkd4\") pod \"network-metrics-daemon-p5wb6\" (UID: \"83354a1c-ecad-424f-8efa-e6bf2e044fae\") " pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:01.050665 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.050643    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8k8b\" (UniqueName: \"kubernetes.io/projected/bfe80ec0-439b-49a1-b189-8d2cfbde3a50-kube-api-access-k8k8b\") pod \"tuned-zv9mn\" (UID: \"bfe80ec0-439b-49a1-b189-8d2cfbde3a50\") " pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.050967 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.050939    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkqsb\" (UniqueName: \"kubernetes.io/projected/c7a095f9-e6d1-40e1-be22-01493dc8f629-kube-api-access-lkqsb\") pod \"ovnkube-node-mtm94\" (UID: \"c7a095f9-e6d1-40e1-be22-01493dc8f629\") " pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.052743 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.052721    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cptxg\" (UniqueName: \"kubernetes.io/projected/652d024a-faff-4a93-bedb-444cc57170b9-kube-api-access-cptxg\") pod \"multus-dpznr\" (UID: \"652d024a-faff-4a93-bedb-444cc57170b9\") " pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.142803 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.142773    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-registration-dir\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.142949 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.142828    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-device-dir\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.142949 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.142898    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-device-dir\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.142949 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.142900    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-registration-dir\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.142949 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.142918    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8b7525af-768b-49bb-9253-fb405d2f393a-iptables-alerter-script\") pod \"iptables-alerter-vb9xp\" (UID: \"8b7525af-768b-49bb-9253-fb405d2f393a\") " pod="openshift-network-operator/iptables-alerter-vb9xp"
Sep 15 11:25:01.143121 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.142951    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/02e080c4-4b79-42c3-9ad1-319b5ca2df4c-konnectivity-ca\") pod \"konnectivity-agent-5sb4n\" (UID: \"02e080c4-4b79-42c3-9ad1-319b5ca2df4c\") " pod="kube-system/konnectivity-agent-5sb4n"
Sep 15 11:25:01.143121 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.142969    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc0a2e63-3309-454b-890d-bc634e4ee43a-host\") pod \"node-ca-xfzbq\" (UID: \"dc0a2e63-3309-454b-890d-bc634e4ee43a\") " pod="openshift-image-registry/node-ca-xfzbq"
Sep 15 11:25:01.143121 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.142995    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25l6v\" (UniqueName: \"kubernetes.io/projected/8b7525af-768b-49bb-9253-fb405d2f393a-kube-api-access-25l6v\") pod \"iptables-alerter-vb9xp\" (UID: \"8b7525af-768b-49bb-9253-fb405d2f393a\") " pod="openshift-network-operator/iptables-alerter-vb9xp"
Sep 15 11:25:01.143121 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143019    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmjjl\" (UniqueName: \"kubernetes.io/projected/dc0a2e63-3309-454b-890d-bc634e4ee43a-kube-api-access-cmjjl\") pod \"node-ca-xfzbq\" (UID: \"dc0a2e63-3309-454b-890d-bc634e4ee43a\") " pod="openshift-image-registry/node-ca-xfzbq"
Sep 15 11:25:01.143121 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143025    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc0a2e63-3309-454b-890d-bc634e4ee43a-host\") pod \"node-ca-xfzbq\" (UID: \"dc0a2e63-3309-454b-890d-bc634e4ee43a\") " pod="openshift-image-registry/node-ca-xfzbq"
Sep 15 11:25:01.143121 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143041    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dc0a2e63-3309-454b-890d-bc634e4ee43a-serviceca\") pod \"node-ca-xfzbq\" (UID: \"dc0a2e63-3309-454b-890d-bc634e4ee43a\") " pod="openshift-image-registry/node-ca-xfzbq"
Sep 15 11:25:01.143121 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143069    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b7525af-768b-49bb-9253-fb405d2f393a-host-slash\") pod \"iptables-alerter-vb9xp\" (UID: \"8b7525af-768b-49bb-9253-fb405d2f393a\") " pod="openshift-network-operator/iptables-alerter-vb9xp"
Sep 15 11:25:01.143121 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143093    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68swh\" (UniqueName: \"kubernetes.io/projected/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-kube-api-access-68swh\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.143382 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143126    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-etc-selinux\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.143382 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143170    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.143382 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143205    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/02e080c4-4b79-42c3-9ad1-319b5ca2df4c-agent-certs\") pod \"konnectivity-agent-5sb4n\" (UID: \"02e080c4-4b79-42c3-9ad1-319b5ca2df4c\") " pod="kube-system/konnectivity-agent-5sb4n"
Sep 15 11:25:01.143382 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143229    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-sys-fs\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.143382 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143255    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-socket-dir\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.143382 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143367    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-socket-dir\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.143630 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143615    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8b7525af-768b-49bb-9253-fb405d2f393a-iptables-alerter-script\") pod \"iptables-alerter-vb9xp\" (UID: \"8b7525af-768b-49bb-9253-fb405d2f393a\") " pod="openshift-network-operator/iptables-alerter-vb9xp"
Sep 15 11:25:01.143665 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143628    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/02e080c4-4b79-42c3-9ad1-319b5ca2df4c-konnectivity-ca\") pod \"konnectivity-agent-5sb4n\" (UID: \"02e080c4-4b79-42c3-9ad1-319b5ca2df4c\") " pod="kube-system/konnectivity-agent-5sb4n"
Sep 15 11:25:01.143698 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143673    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-etc-selinux\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.143698 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143676    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.143775 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143703    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-sys-fs\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.143775 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143725    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8b7525af-768b-49bb-9253-fb405d2f393a-host-slash\") pod \"iptables-alerter-vb9xp\" (UID: \"8b7525af-768b-49bb-9253-fb405d2f393a\") " pod="openshift-network-operator/iptables-alerter-vb9xp"
Sep 15 11:25:01.143863 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.143817    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dc0a2e63-3309-454b-890d-bc634e4ee43a-serviceca\") pod \"node-ca-xfzbq\" (UID: \"dc0a2e63-3309-454b-890d-bc634e4ee43a\") " pod="openshift-image-registry/node-ca-xfzbq"
Sep 15 11:25:01.145878 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.145852    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/02e080c4-4b79-42c3-9ad1-319b5ca2df4c-agent-certs\") pod \"konnectivity-agent-5sb4n\" (UID: \"02e080c4-4b79-42c3-9ad1-319b5ca2df4c\") " pod="kube-system/konnectivity-agent-5sb4n"
Sep 15 11:25:01.160167 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.160122    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68swh\" (UniqueName: \"kubernetes.io/projected/5413bf1b-e2dd-4ee7-b327-e4ac714b0a07-kube-api-access-68swh\") pod \"aws-ebs-csi-driver-node-rwtjs\" (UID: \"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.164288 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.164266    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmjjl\" (UniqueName: \"kubernetes.io/projected/dc0a2e63-3309-454b-890d-bc634e4ee43a-kube-api-access-cmjjl\") pod \"node-ca-xfzbq\" (UID: \"dc0a2e63-3309-454b-890d-bc634e4ee43a\") " pod="openshift-image-registry/node-ca-xfzbq"
Sep 15 11:25:01.164385 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.164308    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25l6v\" (UniqueName: \"kubernetes.io/projected/8b7525af-768b-49bb-9253-fb405d2f393a-kube-api-access-25l6v\") pod \"iptables-alerter-vb9xp\" (UID: \"8b7525af-768b-49bb-9253-fb405d2f393a\") " pod="openshift-network-operator/iptables-alerter-vb9xp"
Sep 15 11:25:01.228264 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.228232    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hq8r8"
Sep 15 11:25:01.237193 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.237170    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zv9mn"
Sep 15 11:25:01.247031 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.247013    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lhdb5"
Sep 15 11:25:01.252909 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.252885    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dpznr"
Sep 15 11:25:01.259691 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.259674    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:01.268472 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.268454    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs"
Sep 15 11:25:01.274412 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.274396    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5sb4n"
Sep 15 11:25:01.282029 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.282013    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vb9xp"
Sep 15 11:25:01.288741 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.288722    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xfzbq"
Sep 15 11:25:01.384022 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.383996    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 11:25:01.546644 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.546549    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:01.546644 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.546639    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs\") pod \"network-metrics-daemon-p5wb6\" (UID: \"83354a1c-ecad-424f-8efa-e6bf2e044fae\") " pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:01.546851 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.546681    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 11:25:01.546851 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.546714    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 11:25:01.546851 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.546757    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret podName:7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b nodeName:}" failed. No retries permitted until 2026-09-15 11:25:02.546738107 +0000 UTC m=+4.039515238 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret") pod "global-pull-secret-syncer-4jcwq" (UID: "7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b") : object "kube-system"/"original-pull-secret" not registered
Sep 15 11:25:01.546851 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.546771    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs podName:83354a1c-ecad-424f-8efa-e6bf2e044fae nodeName:}" failed. No retries permitted until 2026-09-15 11:25:02.546764449 +0000 UTC m=+4.039541576 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs") pod "network-metrics-daemon-p5wb6" (UID: "83354a1c-ecad-424f-8efa-e6bf2e044fae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 11:25:01.647705 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.647674    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwnxg\" (UniqueName: \"kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg\") pod \"network-check-target-sfjzw\" (UID: \"9674a018-9086-45a1-b5cf-ef8e9dcd0366\") " pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:01.647860 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.647840    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 11:25:01.647916 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.647861    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 11:25:01.647916 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.647871    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xwnxg for pod openshift-network-diagnostics/network-check-target-sfjzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 11:25:01.648023 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:01.647922    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg podName:9674a018-9086-45a1-b5cf-ef8e9dcd0366 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:02.647903826 +0000 UTC m=+4.140680964 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xwnxg" (UniqueName: "kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg") pod "network-check-target-sfjzw" (UID: "9674a018-9086-45a1-b5cf-ef8e9dcd0366") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 11:25:01.898915 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:01.898766    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc0a2e63_3309_454b_890d_bc634e4ee43a.slice/crio-26845468e807359308a50b5cf0ac53a905b6995c6de7b98e5b4b1369b2eaffe5 WatchSource:0}: Error finding container 26845468e807359308a50b5cf0ac53a905b6995c6de7b98e5b4b1369b2eaffe5: Status 404 returned error can't find the container with id 26845468e807359308a50b5cf0ac53a905b6995c6de7b98e5b4b1369b2eaffe5
Sep 15 11:25:01.900058 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:01.900031    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bd5c541_ba2b_4ab4_a874_9c0436910724.slice/crio-37f21394fcdcd40cfc64769558c309adfb107f864e6cf53e7a0067d36d63fa6a WatchSource:0}: Error finding container 37f21394fcdcd40cfc64769558c309adfb107f864e6cf53e7a0067d36d63fa6a: Status 404 returned error can't find the container with id 37f21394fcdcd40cfc64769558c309adfb107f864e6cf53e7a0067d36d63fa6a
Sep 15 11:25:01.900669 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:01.900629    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe80ec0_439b_49a1_b189_8d2cfbde3a50.slice/crio-ab59052c6cd1c94e193f556a8540a01bc825911dc2e3b94bd9cc07701ca81d15 WatchSource:0}: Error finding container ab59052c6cd1c94e193f556a8540a01bc825911dc2e3b94bd9cc07701ca81d15: Status 404 returned error can't find the container with id ab59052c6cd1c94e193f556a8540a01bc825911dc2e3b94bd9cc07701ca81d15
Sep 15 11:25:01.902026 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:01.901610    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5413bf1b_e2dd_4ee7_b327_e4ac714b0a07.slice/crio-285ad56519369f850fcf832d991507b910927aeedfc340e152b9c9d1a3aa919a WatchSource:0}: Error finding container 285ad56519369f850fcf832d991507b910927aeedfc340e152b9c9d1a3aa919a: Status 404 returned error can't find the container with id 285ad56519369f850fcf832d991507b910927aeedfc340e152b9c9d1a3aa919a
Sep 15 11:25:01.904782 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:01.904747    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7a095f9_e6d1_40e1_be22_01493dc8f629.slice/crio-8e9cf894d16736e361bf610aab99dbd0c28b40573ade60bca84a29e32bcd4b85 WatchSource:0}: Error finding container 8e9cf894d16736e361bf610aab99dbd0c28b40573ade60bca84a29e32bcd4b85: Status 404 returned error can't find the container with id 8e9cf894d16736e361bf610aab99dbd0c28b40573ade60bca84a29e32bcd4b85
Sep 15 11:25:01.905672 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:01.905642    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02e080c4_4b79_42c3_9ad1_319b5ca2df4c.slice/crio-3817c88e16e8fb52f1f6f734421809b6ea02e86f2357dc08445957e6d425a966 WatchSource:0}: Error finding container 3817c88e16e8fb52f1f6f734421809b6ea02e86f2357dc08445957e6d425a966: Status 404 returned error can't find the container with id 3817c88e16e8fb52f1f6f734421809b6ea02e86f2357dc08445957e6d425a966
Sep 15 11:25:01.906221 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:01.906199    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43830c40_9ac0_4143_b673_a73a6273075c.slice/crio-9b99abc66061566626c816824bba8b7270b5c7e62d33b4e190e974fb09f2b80a WatchSource:0}: Error finding container 9b99abc66061566626c816824bba8b7270b5c7e62d33b4e190e974fb09f2b80a: Status 404 returned error can't find the container with id 9b99abc66061566626c816824bba8b7270b5c7e62d33b4e190e974fb09f2b80a
Sep 15 11:25:01.907274 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:01.907172    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod652d024a_faff_4a93_bedb_444cc57170b9.slice/crio-452ab78afba0dbdb8959d086dadbd86fa9a0541e7e92d9280b8b3af1250f3c58 WatchSource:0}: Error finding container 452ab78afba0dbdb8959d086dadbd86fa9a0541e7e92d9280b8b3af1250f3c58: Status 404 returned error can't find the container with id 452ab78afba0dbdb8959d086dadbd86fa9a0541e7e92d9280b8b3af1250f3c58
Sep 15 11:25:01.971156 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.971101    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 11:20:00 +0000 UTC" deadline="2028-05-31 04:49:16.18480326 +0000 UTC"
Sep 15 11:25:01.971156 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:01.971154    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14969h24m14.213653328s"
Sep 15 11:25:02.028455 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.028426    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:02.028611 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:02.028524    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:02.042388 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.042356    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-153.ec2.internal" event={"ID":"1142610769d3e76a0d4f81b8e01ff4c0","Type":"ContainerStarted","Data":"46bd04bd6ca88b818b5bb747c1dbb8e1b9e19363a7d9fae8d3d98215da4ffa45"}
Sep 15 11:25:02.043503 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.043477    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dpznr" event={"ID":"652d024a-faff-4a93-bedb-444cc57170b9","Type":"ContainerStarted","Data":"452ab78afba0dbdb8959d086dadbd86fa9a0541e7e92d9280b8b3af1250f3c58"}
Sep 15 11:25:02.044514 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.044491    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hq8r8" event={"ID":"43830c40-9ac0-4143-b673-a73a6273075c","Type":"ContainerStarted","Data":"9b99abc66061566626c816824bba8b7270b5c7e62d33b4e190e974fb09f2b80a"}
Sep 15 11:25:02.048765 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.048745    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94" event={"ID":"c7a095f9-e6d1-40e1-be22-01493dc8f629","Type":"ContainerStarted","Data":"8e9cf894d16736e361bf610aab99dbd0c28b40573ade60bca84a29e32bcd4b85"}
Sep 15 11:25:02.051929 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.051910    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zv9mn" event={"ID":"bfe80ec0-439b-49a1-b189-8d2cfbde3a50","Type":"ContainerStarted","Data":"ab59052c6cd1c94e193f556a8540a01bc825911dc2e3b94bd9cc07701ca81d15"}
Sep 15 11:25:02.052993 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.052969    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xfzbq" event={"ID":"dc0a2e63-3309-454b-890d-bc634e4ee43a","Type":"ContainerStarted","Data":"26845468e807359308a50b5cf0ac53a905b6995c6de7b98e5b4b1369b2eaffe5"}
Sep 15 11:25:02.054825 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.054804    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vb9xp" event={"ID":"8b7525af-768b-49bb-9253-fb405d2f393a","Type":"ContainerStarted","Data":"19be7ed9aee71e058379067d26b980ae2f2700d7adf503b860da9eb712c3f425"}
Sep 15 11:25:02.056967 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.056933    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-153.ec2.internal" podStartSLOduration=2.056921307 podStartE2EDuration="2.056921307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 11:25:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:02.056850639 +0000 UTC m=+3.549627800" watchObservedRunningTime="2026-09-15 11:25:02.056921307 +0000 UTC m=+3.549698452"
Sep 15 11:25:02.057538 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.057513    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5sb4n" event={"ID":"02e080c4-4b79-42c3-9ad1-319b5ca2df4c","Type":"ContainerStarted","Data":"3817c88e16e8fb52f1f6f734421809b6ea02e86f2357dc08445957e6d425a966"}
Sep 15 11:25:02.059011 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.058956    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs" event={"ID":"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07","Type":"ContainerStarted","Data":"285ad56519369f850fcf832d991507b910927aeedfc340e152b9c9d1a3aa919a"}
Sep 15 11:25:02.060178 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.060157    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhdb5" event={"ID":"1bd5c541-ba2b-4ab4-a874-9c0436910724","Type":"ContainerStarted","Data":"37f21394fcdcd40cfc64769558c309adfb107f864e6cf53e7a0067d36d63fa6a"}
Sep 15 11:25:02.553961 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.553882    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs\") pod \"network-metrics-daemon-p5wb6\" (UID: \"83354a1c-ecad-424f-8efa-e6bf2e044fae\") " pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:02.553961 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.553942    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:02.554162 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:02.554078    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 11:25:02.554162 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:02.554138    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret podName:7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b nodeName:}" failed. No retries permitted until 2026-09-15 11:25:04.554120313 +0000 UTC m=+6.046897453 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret") pod "global-pull-secret-syncer-4jcwq" (UID: "7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b") : object "kube-system"/"original-pull-secret" not registered
Sep 15 11:25:02.554552 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:02.554527    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 11:25:02.554679 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:02.554600    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs podName:83354a1c-ecad-424f-8efa-e6bf2e044fae nodeName:}" failed. No retries permitted until 2026-09-15 11:25:04.554585878 +0000 UTC m=+6.047363007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs") pod "network-metrics-daemon-p5wb6" (UID: "83354a1c-ecad-424f-8efa-e6bf2e044fae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 11:25:02.655669 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.655077    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwnxg\" (UniqueName: \"kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg\") pod \"network-check-target-sfjzw\" (UID: \"9674a018-9086-45a1-b5cf-ef8e9dcd0366\") " pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:02.655669 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:02.655246    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 11:25:02.655669 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:02.655268    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 11:25:02.655669 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:02.655281    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xwnxg for pod openshift-network-diagnostics/network-check-target-sfjzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 11:25:02.655669 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:02.655335    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg podName:9674a018-9086-45a1-b5cf-ef8e9dcd0366 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:04.655318847 +0000 UTC m=+6.148095977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xwnxg" (UniqueName: "kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg") pod "network-check-target-sfjzw" (UID: "9674a018-9086-45a1-b5cf-ef8e9dcd0366") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 11:25:02.971604 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.971500    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 11:20:00 +0000 UTC" deadline="2028-04-18 04:21:29.623600045 +0000 UTC"
Sep 15 11:25:02.971604 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:02.971539    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13936h56m26.652066031s"
Sep 15 11:25:03.030517 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:03.030487    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:03.030677 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:03.030645    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:03.031305 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:03.031267    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:03.031391 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:03.031368    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:03.066685 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:03.066655    2560 generic.go:356] "Generic (PLEG): container finished" podID="f7ed78fa2944f8540ed193d79ae916e7" containerID="d6dfd292b98ebb6048862a878436bd420daffb21a481d2edf6dfa7d3d16480ce" exitCode=0
Sep 15 11:25:03.067163 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:03.067140    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal" event={"ID":"f7ed78fa2944f8540ed193d79ae916e7","Type":"ContainerDied","Data":"d6dfd292b98ebb6048862a878436bd420daffb21a481d2edf6dfa7d3d16480ce"}
Sep 15 11:25:04.028439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:04.027966    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:04.028439 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:04.028107    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:04.082110 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:04.082017    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal" event={"ID":"f7ed78fa2944f8540ed193d79ae916e7","Type":"ContainerStarted","Data":"21270846ef58fda7399b8ce551d1ae076f8122a1fd35217d7f7aa182a65d6f00"}
Sep 15 11:25:04.096082 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:04.095900    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-153.ec2.internal" podStartSLOduration=4.095879868 podStartE2EDuration="4.095879868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 11:25:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:04.095089796 +0000 UTC m=+5.587866944" watchObservedRunningTime="2026-09-15 11:25:04.095879868 +0000 UTC m=+5.588657017"
Sep 15 11:25:04.571399 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:04.570598    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:04.571399 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:04.570683    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs\") pod \"network-metrics-daemon-p5wb6\" (UID: \"83354a1c-ecad-424f-8efa-e6bf2e044fae\") " pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:04.571399 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:04.570809    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 11:25:04.571399 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:04.570870    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs podName:83354a1c-ecad-424f-8efa-e6bf2e044fae nodeName:}" failed. No retries permitted until 2026-09-15 11:25:08.570851245 +0000 UTC m=+10.063628385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs") pod "network-metrics-daemon-p5wb6" (UID: "83354a1c-ecad-424f-8efa-e6bf2e044fae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 11:25:04.571399 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:04.571246    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 11:25:04.571399 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:04.571298    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret podName:7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b nodeName:}" failed. No retries permitted until 2026-09-15 11:25:08.571282773 +0000 UTC m=+10.064059904 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret") pod "global-pull-secret-syncer-4jcwq" (UID: "7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b") : object "kube-system"/"original-pull-secret" not registered
Sep 15 11:25:04.671447 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:04.671412    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwnxg\" (UniqueName: \"kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg\") pod \"network-check-target-sfjzw\" (UID: \"9674a018-9086-45a1-b5cf-ef8e9dcd0366\") " pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:04.672038 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:04.671619    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 11:25:04.672038 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:04.671643    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 11:25:04.672038 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:04.671656    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xwnxg for pod openshift-network-diagnostics/network-check-target-sfjzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 11:25:04.672038 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:04.671710    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg podName:9674a018-9086-45a1-b5cf-ef8e9dcd0366 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:08.671691723 +0000 UTC m=+10.164468849 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xwnxg" (UniqueName: "kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg") pod "network-check-target-sfjzw" (UID: "9674a018-9086-45a1-b5cf-ef8e9dcd0366") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 11:25:05.028728 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:05.028053    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:05.028728 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:05.028176    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:05.028728 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:05.028053    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:05.028728 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:05.028623    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:06.029414 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:06.029117    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:06.029414 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:06.029224    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:07.029852 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:07.029207    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:07.029852 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:07.029332    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:07.029852 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:07.029696    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:07.029852 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:07.029799    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:08.027842 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:08.027809    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:08.028026 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:08.027983    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:08.605979 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:08.605934    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:08.606456 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:08.606034    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs\") pod \"network-metrics-daemon-p5wb6\" (UID: \"83354a1c-ecad-424f-8efa-e6bf2e044fae\") " pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:08.606456 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:08.606169    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 11:25:08.606456 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:08.606230    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs podName:83354a1c-ecad-424f-8efa-e6bf2e044fae nodeName:}" failed. No retries permitted until 2026-09-15 11:25:16.606211535 +0000 UTC m=+18.098988663 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs") pod "network-metrics-daemon-p5wb6" (UID: "83354a1c-ecad-424f-8efa-e6bf2e044fae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 11:25:08.606681 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:08.606533    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 11:25:08.606681 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:08.606615    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret podName:7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b nodeName:}" failed. No retries permitted until 2026-09-15 11:25:16.606594738 +0000 UTC m=+18.099371877 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret") pod "global-pull-secret-syncer-4jcwq" (UID: "7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b") : object "kube-system"/"original-pull-secret" not registered
Sep 15 11:25:08.707960 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:08.707382    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwnxg\" (UniqueName: \"kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg\") pod \"network-check-target-sfjzw\" (UID: \"9674a018-9086-45a1-b5cf-ef8e9dcd0366\") " pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:08.707960 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:08.707524    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 11:25:08.707960 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:08.707547    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 11:25:08.707960 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:08.707575    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xwnxg for pod openshift-network-diagnostics/network-check-target-sfjzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 11:25:08.707960 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:08.707636    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg podName:9674a018-9086-45a1-b5cf-ef8e9dcd0366 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:16.707615781 +0000 UTC m=+18.200392926 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xwnxg" (UniqueName: "kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg") pod "network-check-target-sfjzw" (UID: "9674a018-9086-45a1-b5cf-ef8e9dcd0366") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 11:25:09.029671 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:09.029603    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:09.029806 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:09.029692    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:09.030077 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:09.030056    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:09.030179 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:09.030155    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:10.029284 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:10.028773    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:10.029284 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:10.028919    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:11.028285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:11.028248    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:11.028446 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:11.028349    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:11.028446 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:11.028376    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:11.028446 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:11.028430    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:12.028315 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:12.028287    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:12.028707 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:12.028383    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:13.028387 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:13.028354    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:13.028387 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:13.028371    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:13.028908 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:13.028486    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:13.028908 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:13.028658    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:14.027902 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:14.027875    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:14.028084 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:14.027974    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:15.028045 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:15.028007    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:15.028435 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:15.028010    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:15.028435 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:15.028160    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:15.028435 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:15.028186    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:16.028723 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:16.028683    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:16.029152 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:16.028817    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:16.664115 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:16.664056    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs\") pod \"network-metrics-daemon-p5wb6\" (UID: \"83354a1c-ecad-424f-8efa-e6bf2e044fae\") " pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:16.664286 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:16.664146    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:16.664286 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:16.664228    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 11:25:16.664404 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:16.664237    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 11:25:16.664404 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:16.664298    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs podName:83354a1c-ecad-424f-8efa-e6bf2e044fae nodeName:}" failed. No retries permitted until 2026-09-15 11:25:32.664277852 +0000 UTC m=+34.157054987 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs") pod "network-metrics-daemon-p5wb6" (UID: "83354a1c-ecad-424f-8efa-e6bf2e044fae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 11:25:16.664404 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:16.664351    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret podName:7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b nodeName:}" failed. No retries permitted until 2026-09-15 11:25:32.664329905 +0000 UTC m=+34.157107057 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret") pod "global-pull-secret-syncer-4jcwq" (UID: "7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b") : object "kube-system"/"original-pull-secret" not registered
Sep 15 11:25:16.765025 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:16.764990    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwnxg\" (UniqueName: \"kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg\") pod \"network-check-target-sfjzw\" (UID: \"9674a018-9086-45a1-b5cf-ef8e9dcd0366\") " pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:16.765216 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:16.765141    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 11:25:16.765216 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:16.765166    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 11:25:16.765216 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:16.765178    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xwnxg for pod openshift-network-diagnostics/network-check-target-sfjzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 11:25:16.765365 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:16.765238    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg podName:9674a018-9086-45a1-b5cf-ef8e9dcd0366 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:32.765220395 +0000 UTC m=+34.257997521 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xwnxg" (UniqueName: "kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg") pod "network-check-target-sfjzw" (UID: "9674a018-9086-45a1-b5cf-ef8e9dcd0366") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 11:25:17.028009 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:17.027905    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:17.028009 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:17.027937    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:17.028201 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:17.028030    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:17.028201 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:17.028154    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:18.028247 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:18.028203    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:18.028678 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:18.028339    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:19.029274 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:19.029238    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:19.029707 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:19.029343    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:19.029707 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:19.029434    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:19.029707 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:19.029551    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:20.028112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.028044    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:20.028193 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:20.028171    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:20.114818 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.114787    2560 generic.go:356] "Generic (PLEG): container finished" podID="1bd5c541-ba2b-4ab4-a874-9c0436910724" containerID="7f7d5d31dc0b1fc2c246b90e65345fb70071e144c37b3cb53c2c3b2396bbaa2f" exitCode=0
Sep 15 11:25:20.115348 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.114890    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhdb5" event={"ID":"1bd5c541-ba2b-4ab4-a874-9c0436910724","Type":"ContainerDied","Data":"7f7d5d31dc0b1fc2c246b90e65345fb70071e144c37b3cb53c2c3b2396bbaa2f"}
Sep 15 11:25:20.117101 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.116651    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dpznr" event={"ID":"652d024a-faff-4a93-bedb-444cc57170b9","Type":"ContainerStarted","Data":"b87cdd3f9c2d76d02c31129b0076fc70baf70759765664a9927b6a00ad080f90"}
Sep 15 11:25:20.118610 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.118590    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hq8r8" event={"ID":"43830c40-9ac0-4143-b673-a73a6273075c","Type":"ContainerStarted","Data":"436a80f42f556872f1fe941a9f15a49be91d365543c62f1139854be3f58fe1d6"}
Sep 15 11:25:20.121082 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.121054    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94" event={"ID":"c7a095f9-e6d1-40e1-be22-01493dc8f629","Type":"ContainerStarted","Data":"a12a9860cc87d72c546c6f5a4162b8fa907adf51ab5247516de453a318fd3754"}
Sep 15 11:25:20.121175 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.121148    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94" event={"ID":"c7a095f9-e6d1-40e1-be22-01493dc8f629","Type":"ContainerStarted","Data":"9a7c2a01f247f377618aeee697d5e633ac02fa5a593bc00897302fe50ccf4858"}
Sep 15 11:25:20.121218 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.121202    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94" event={"ID":"c7a095f9-e6d1-40e1-be22-01493dc8f629","Type":"ContainerStarted","Data":"5bbd673c4bd2936971cdcd829393f7d68d714e93465fa725c523c201af036bd1"}
Sep 15 11:25:20.123194 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.123168    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zv9mn" event={"ID":"bfe80ec0-439b-49a1-b189-8d2cfbde3a50","Type":"ContainerStarted","Data":"292dfc7f1f8e40dbd14ee013fb0ab1f8ff8514d93ea5dbc0a73ad4a1fd9efbc5"}
Sep 15 11:25:20.124885 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.124861    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xfzbq" event={"ID":"dc0a2e63-3309-454b-890d-bc634e4ee43a","Type":"ContainerStarted","Data":"1376253f1bdcc1ee6d7b4a1e4b024a52c86479cd81494e301349c3311b19ef13"}
Sep 15 11:25:20.126426 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.126405    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5sb4n" event={"ID":"02e080c4-4b79-42c3-9ad1-319b5ca2df4c","Type":"ContainerStarted","Data":"faa528cc8f635c9f28c587bcc3c56ccb78b6d07c9ae43b35120a1c9e9a77dadb"}
Sep 15 11:25:20.127805 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.127786    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs" event={"ID":"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07","Type":"ContainerStarted","Data":"7ec70e8139750b486096a9cf17d51d2b81bf7e7ec901aabde8a804fb9d9f756b"}
Sep 15 11:25:20.153534 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.153486    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xfzbq" podStartSLOduration=3.368585983 podStartE2EDuration="21.153470094s" totalImagesPullingTime="17.784884111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 11:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:20.149790974 +0000 UTC m=+21.642568124" watchObservedRunningTime="2026-09-15 11:25:20.153470094 +0000 UTC m=+21.646247243"
Sep 15 11:25:20.169334 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.169294    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zv9mn" podStartSLOduration=3.354413809 podStartE2EDuration="21.169280572s" totalImagesPullingTime="17.814866763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 11:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:20.169057422 +0000 UTC m=+21.661834570" watchObservedRunningTime="2026-09-15 11:25:20.169280572 +0000 UTC m=+21.662057720"
Sep 15 11:25:20.184179 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.184146    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5sb4n" podStartSLOduration=3.416642723 podStartE2EDuration="21.184133546s" totalImagesPullingTime="17.767490823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:20.183831465 +0000 UTC m=+21.676608613" watchObservedRunningTime="2026-09-15 11:25:20.184133546 +0000 UTC m=+21.676910694"
Sep 15 11:25:20.237863 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.237823    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dpznr" podStartSLOduration=3.306513018 podStartE2EDuration="21.237809046s" totalImagesPullingTime="17.931296028s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 11:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:20.237758411 +0000 UTC m=+21.730535560" watchObservedRunningTime="2026-09-15 11:25:20.237809046 +0000 UTC m=+21.730586188"
Sep 15 11:25:20.238083 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.238064    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hq8r8" podStartSLOduration=3.430536792 podStartE2EDuration="21.238060005s" totalImagesPullingTime="17.807523213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 11:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:20.207945434 +0000 UTC m=+21.700722583" watchObservedRunningTime="2026-09-15 11:25:20.238060005 +0000 UTC m=+21.730837152"
Sep 15 11:25:20.813736 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.813705    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 15 11:25:20.989161 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.989067    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T11:25:20.813758249Z","UUID":"a29f0082-3e72-418a-bf75-ad6821723f33","Handler":null,"Name":"","Endpoint":""}
Sep 15 11:25:20.990584 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.990549    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 15 11:25:20.990713 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:20.990591    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 15 11:25:21.028360 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:21.028334    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:21.028503 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:21.028448    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:21.028503 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:21.028483    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:21.028650 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:21.028586    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:21.133909 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:21.133877    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94" event={"ID":"c7a095f9-e6d1-40e1-be22-01493dc8f629","Type":"ContainerStarted","Data":"1b928a793fb2a7349c58f635f6f8fb69b66c5b7a00756157dde87fd6613cc249"}
Sep 15 11:25:21.134398 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:21.133966    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94" event={"ID":"c7a095f9-e6d1-40e1-be22-01493dc8f629","Type":"ContainerStarted","Data":"46b0e5912102b5a55f5f6671d2591eff577146b8e39472c3c468596d952963b2"}
Sep 15 11:25:21.134398 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:21.134018    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94" event={"ID":"c7a095f9-e6d1-40e1-be22-01493dc8f629","Type":"ContainerStarted","Data":"14dffc9fdf21c1262af0c189a58e7159c91e5981c7a2eda1ae94eadfd64b519d"}
Sep 15 11:25:21.136008 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:21.135981    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vb9xp" event={"ID":"8b7525af-768b-49bb-9253-fb405d2f393a","Type":"ContainerStarted","Data":"d601115b67a381228bd546ae886459f0b0743c706474213d92f6a0d618c7264e"}
Sep 15 11:25:21.138071 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:21.138036    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs" event={"ID":"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07","Type":"ContainerStarted","Data":"8244e992d7765544df96dce6c576d82f06d9b53bebfae3ef7e5be88fe23db4c2"}
Sep 15 11:25:21.155793 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:21.155744    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vb9xp" podStartSLOduration=4.350874654 podStartE2EDuration="22.155726691s" totalImagesPullingTime="17.804852037s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 11:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:21.155533624 +0000 UTC m=+22.648310772" watchObservedRunningTime="2026-09-15 11:25:21.155726691 +0000 UTC m=+22.648503841"
Sep 15 11:25:21.201978 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:21.201765    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5sb4n"
Sep 15 11:25:22.028859 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:22.028781    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:22.029021 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:22.028890    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:22.143193 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:22.143155    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs" event={"ID":"5413bf1b-e2dd-4ee7-b327-e4ac714b0a07","Type":"ContainerStarted","Data":"9044dc50de0a1e1b386849af6b20e6e01a2214ce3f890a02e78f529f5aa826e0"}
Sep 15 11:25:22.162330 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:22.162282    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rwtjs" podStartSLOduration=3.784606561 podStartE2EDuration="23.162267341s" totalImagesPullingTime="19.37766078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 11:24:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:22.161802519 +0000 UTC m=+23.654579667" watchObservedRunningTime="2026-09-15 11:25:22.162267341 +0000 UTC m=+23.655044486"
Sep 15 11:25:23.028554 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:23.028522    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:23.028743 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:23.028582    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:23.028743 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:23.028667    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:23.028862 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:23.028776    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:23.149927 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:23.149889    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94" event={"ID":"c7a095f9-e6d1-40e1-be22-01493dc8f629","Type":"ContainerStarted","Data":"98347f483dc12e362b4e6e5aaa5339fd94079e83bcbdd49bc165de58dde3cd76"}
Sep 15 11:25:24.028369 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:24.028336    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:24.028881 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:24.028844    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:24.729854 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:24.729621    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5sb4n"
Sep 15 11:25:24.730266 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:24.730159    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5sb4n"
Sep 15 11:25:25.028642 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:25.028543    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:25.028926 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:25.028681    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:25.028926 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:25.028717    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:25.028926 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:25.028816    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:25.156111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:25.156076    2560 generic.go:356] "Generic (PLEG): container finished" podID="1bd5c541-ba2b-4ab4-a874-9c0436910724" containerID="f3485fbb6d0f1e69883c3889b48e85ed92decb610ff967e70901253330438e15" exitCode=0
Sep 15 11:25:25.156279 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:25.156152    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhdb5" event={"ID":"1bd5c541-ba2b-4ab4-a874-9c0436910724","Type":"ContainerDied","Data":"f3485fbb6d0f1e69883c3889b48e85ed92decb610ff967e70901253330438e15"}
Sep 15 11:25:25.160370 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:25.160332    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94" event={"ID":"c7a095f9-e6d1-40e1-be22-01493dc8f629","Type":"ContainerStarted","Data":"56e1bc578fb0b836a6e3c77ec99e29753c5cdcb3b0c99907038091b9abc75785"}
Sep 15 11:25:25.160719 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:25.160693    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:25.160803 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:25.160778    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:25.160851 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:25.160827    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:25.161036 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:25.161017    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5sb4n"
Sep 15 11:25:25.175317 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:25.175298    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:25.175434 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:25.175425    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:25.232975 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:25.232938    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94" podStartSLOduration=8.409526914 podStartE2EDuration="26.232925486s" totalImagesPullingTime="17.823398572s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 11:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:25.231898565 +0000 UTC m=+26.724675727" watchObservedRunningTime="2026-09-15 11:25:25.232925486 +0000 UTC m=+26.725702625"
Sep 15 11:25:26.028101 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:26.028076    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:26.028577 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:26.028168    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:26.164223 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:26.164188    2560 generic.go:356] "Generic (PLEG): container finished" podID="1bd5c541-ba2b-4ab4-a874-9c0436910724" containerID="b44eb9ebe1d94687baa6418b0825b3e57a0374d99de21c2542dd981d22599fef" exitCode=0
Sep 15 11:25:26.164402 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:26.164270    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhdb5" event={"ID":"1bd5c541-ba2b-4ab4-a874-9c0436910724","Type":"ContainerDied","Data":"b44eb9ebe1d94687baa6418b0825b3e57a0374d99de21c2542dd981d22599fef"}
Sep 15 11:25:26.380294 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:26.380263    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sfjzw"]
Sep 15 11:25:26.380461 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:26.380396    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:26.380529 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:26.380508    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:26.382854 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:26.382831    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4jcwq"]
Sep 15 11:25:26.382951 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:26.382903    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:26.382998 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:26.382965    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:26.385797 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:26.385776    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p5wb6"]
Sep 15 11:25:26.385884 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:26.385857    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:26.385935 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:26.385921    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:27.168777 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:27.168746    2560 generic.go:356] "Generic (PLEG): container finished" podID="1bd5c541-ba2b-4ab4-a874-9c0436910724" containerID="7f57d9f1f87bf2de972ea201d3334ab3fbf1e5d44e4ef59f8b910b4283ae5532" exitCode=0
Sep 15 11:25:27.169144 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:27.168828    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhdb5" event={"ID":"1bd5c541-ba2b-4ab4-a874-9c0436910724","Type":"ContainerDied","Data":"7f57d9f1f87bf2de972ea201d3334ab3fbf1e5d44e4ef59f8b910b4283ae5532"}
Sep 15 11:25:28.028553 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:28.028517    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:28.028714 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:28.028631    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:28.028714 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:28.028637    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:28.028714 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:28.028659    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:28.028823 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:28.028735    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:28.028823 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:28.028804    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:30.028418 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:30.028194    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:30.028908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:30.028280    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:30.028908 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:30.028504    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:30.028908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:30.028297    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:30.028908 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:30.028608    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:30.028908 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:30.028696    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:32.028850 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.028817    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:32.028850 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.028835    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:32.029327 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.028817    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:32.029327 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:32.028945    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4jcwq" podUID="7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b"
Sep 15 11:25:32.029327 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:32.029071    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p5wb6" podUID="83354a1c-ecad-424f-8efa-e6bf2e044fae"
Sep 15 11:25:32.029327 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:32.029151    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfjzw" podUID="9674a018-9086-45a1-b5cf-ef8e9dcd0366"
Sep 15 11:25:32.675849 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.675818    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs\") pod \"network-metrics-daemon-p5wb6\" (UID: \"83354a1c-ecad-424f-8efa-e6bf2e044fae\") " pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:32.676018 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.675861    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:32.676018 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:32.675969    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 11:25:32.676090 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:32.676023    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret podName:7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b nodeName:}" failed. No retries permitted until 2026-09-15 11:26:04.676010623 +0000 UTC m=+66.168787762 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret") pod "global-pull-secret-syncer-4jcwq" (UID: "7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b") : object "kube-system"/"original-pull-secret" not registered
Sep 15 11:25:32.676090 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:32.675969    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 11:25:32.676158 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:32.676101    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs podName:83354a1c-ecad-424f-8efa-e6bf2e044fae nodeName:}" failed. No retries permitted until 2026-09-15 11:26:04.676089036 +0000 UTC m=+66.168866169 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs") pod "network-metrics-daemon-p5wb6" (UID: "83354a1c-ecad-424f-8efa-e6bf2e044fae") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 11:25:32.776896 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.776867    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwnxg\" (UniqueName: \"kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg\") pod \"network-check-target-sfjzw\" (UID: \"9674a018-9086-45a1-b5cf-ef8e9dcd0366\") " pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:32.777032 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:32.777018    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 11:25:32.777072 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:32.777038    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 11:25:32.777072 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:32.777048    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xwnxg for pod openshift-network-diagnostics/network-check-target-sfjzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 11:25:32.777132 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:32.777091    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg podName:9674a018-9086-45a1-b5cf-ef8e9dcd0366 nodeName:}" failed. No retries permitted until 2026-09-15 11:26:04.777079941 +0000 UTC m=+66.269857072 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xwnxg" (UniqueName: "kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg") pod "network-check-target-sfjzw" (UID: "9674a018-9086-45a1-b5cf-ef8e9dcd0366") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 11:25:32.806342 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.806320    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-153.ec2.internal" event="NodeReady"
Sep 15 11:25:32.806451 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.806416    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 15 11:25:32.840584 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.840546    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-784668b6-dtn6x"]
Sep 15 11:25:32.870821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.870797    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-njsmc"]
Sep 15 11:25:32.870952 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.870931    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:32.873184 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.873162    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 15 11:25:32.873393 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.873379    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xsd55\""
Sep 15 11:25:32.873542 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.873510    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 15 11:25:32.873808 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.873795    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 15 11:25:32.885478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.885454    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 15 11:25:32.895144 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.895122    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"]
Sep 15 11:25:32.895256 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.895240    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:32.897752 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.897735    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 15 11:25:32.898255 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.898240    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 15 11:25:32.898447 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.898293    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4gq4s\""
Sep 15 11:25:32.898718 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.898702    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 15 11:25:32.900016 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.899999    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 15 11:25:32.909356 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.909282    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z"]
Sep 15 11:25:32.909450 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.909403    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"
Sep 15 11:25:32.915671 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.915654    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 15 11:25:32.915771 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.915714    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 15 11:25:32.915839 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.915804    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 15 11:25:32.916637 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.916619    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wl5v5\""
Sep 15 11:25:32.922108 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.922093    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 15 11:25:32.936395 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.936357    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-568d798994-hm7cl"]
Sep 15 11:25:32.936472 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.936458    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z"
Sep 15 11:25:32.938335 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.938318    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 15 11:25:32.938335 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.938333    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 15 11:25:32.938480 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.938316    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 15 11:25:32.938892 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.938858    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 15 11:25:32.939063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.939044    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-5drqm\""
Sep 15 11:25:32.954450 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.954429    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-6rnxm"]
Sep 15 11:25:32.954573 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.954547    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:32.956375 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.956353    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 15 11:25:32.956466 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.956440    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 15 11:25:32.956665 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.956646    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 15 11:25:32.956772 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.956648    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 15 11:25:32.956772 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.956689    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 15 11:25:32.956772 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.956751    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 15 11:25:32.956949 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.956931    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-p5slf\""
Sep 15 11:25:32.973684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.973630    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5"]
Sep 15 11:25:32.973827 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.973810    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-6rnxm"
Sep 15 11:25:32.975582 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.975547    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zwk92\""
Sep 15 11:25:32.975680 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.975550    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 15 11:25:32.975680 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.975650    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 15 11:25:32.978326 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978302    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e93e581e-424f-4960-b2e9-fa94c29f0779-image-registry-private-configuration\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:32.978398 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978345    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:32.978438 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978394    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2667a7e4-1546-43a1-b61d-01f8eefe3c82-snapshots\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:32.978438 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978420    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e93e581e-424f-4960-b2e9-fa94c29f0779-installation-pull-secrets\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:32.978503 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978448    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2667a7e4-1546-43a1-b61d-01f8eefe3c82-trusted-ca-bundle\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:32.978503 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978472    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-n7sgg\" (UID: \"e1520a65-2dce-41ce-be10-94731499c0bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"
Sep 15 11:25:32.978503 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978495    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2667a7e4-1546-43a1-b61d-01f8eefe3c82-service-ca-bundle\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:32.978625 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978518    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-bound-sa-token\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:32.978625 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978541    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2667a7e4-1546-43a1-b61d-01f8eefe3c82-tmp\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:32.978625 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978597    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-certificates\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:32.978740 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978647    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2667a7e4-1546-43a1-b61d-01f8eefe3c82-serving-cert\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:32.978740 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978718    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sc2v\" (UniqueName: \"kubernetes.io/projected/2667a7e4-1546-43a1-b61d-01f8eefe3c82-kube-api-access-4sc2v\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:32.978815 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978786    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e93e581e-424f-4960-b2e9-fa94c29f0779-ca-trust-extracted\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:32.978859 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978837    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvzbf\" (UniqueName: \"kubernetes.io/projected/e1520a65-2dce-41ce-be10-94731499c0bd-kube-api-access-rvzbf\") pod \"cluster-samples-operator-86499c89c4-n7sgg\" (UID: \"e1520a65-2dce-41ce-be10-94731499c0bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"
Sep 15 11:25:32.978897 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978863    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e93e581e-424f-4960-b2e9-fa94c29f0779-trusted-ca\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:32.978897 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.978886    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66vdv\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-kube-api-access-66vdv\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:32.992912 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.992896    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-dw8hm"]
Sep 15 11:25:32.993025 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.993009    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5"
Sep 15 11:25:32.995027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.995009    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 15 11:25:32.995177 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.995031    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 15 11:25:32.995352 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.995223    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 15 11:25:32.995352 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.995288    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 15 11:25:32.995546 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:32.995531    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7fp4f\""
Sep 15 11:25:33.021837 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.021787    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"]
Sep 15 11:25:33.022027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.022002    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:33.024120 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.024098    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 15 11:25:33.024365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.024350    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 15 11:25:33.024773 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.024752    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 15 11:25:33.024850 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.024799    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 15 11:25:33.024889 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.024862    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-mtnsq\""
Sep 15 11:25:33.029190 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.029170    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 15 11:25:33.056200 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.056180    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:25:33.059470 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.059443    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 15 11:25:33.059623 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.059444    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 15 11:25:33.059725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.059623    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nnx28\""
Sep 15 11:25:33.059725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.059689    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 15 11:25:33.059915 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.059757    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 15 11:25:33.059915 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.059796    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-c4gb7"]
Sep 15 11:25:33.079984 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.079965    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nvm6\" (UniqueName: \"kubernetes.io/projected/a98bf294-4c61-4611-959d-e4941241a01b-kube-api-access-9nvm6\") pod \"volume-data-source-validator-86695b7cc7-6rnxm\" (UID: \"a98bf294-4c61-4611-959d-e4941241a01b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-6rnxm"
Sep 15 11:25:33.080065 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.079993    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e93e581e-424f-4960-b2e9-fa94c29f0779-ca-trust-extracted\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.080065 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.080009    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:33.080065 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.080031    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvxf2\" (UniqueName: \"kubernetes.io/projected/c8329b83-30a0-4ead-9c95-57f0916610aa-kube-api-access-jvxf2\") pod \"service-ca-operator-cf7c776b7-n5zv5\" (UID: \"c8329b83-30a0-4ead-9c95-57f0916610aa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5"
Sep 15 11:25:33.080179 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.080074    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvzbf\" (UniqueName: \"kubernetes.io/projected/e1520a65-2dce-41ce-be10-94731499c0bd-kube-api-access-rvzbf\") pod \"cluster-samples-operator-86499c89c4-n7sgg\" (UID: \"e1520a65-2dce-41ce-be10-94731499c0bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"
Sep 15 11:25:33.080179 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.080107    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e93e581e-424f-4960-b2e9-fa94c29f0779-trusted-ca\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.080179 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.080126    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66vdv\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-kube-api-access-66vdv\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.080323 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.080208    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e93e581e-424f-4960-b2e9-fa94c29f0779-image-registry-private-configuration\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.080323 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.080244    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.080323 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.080303    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2667a7e4-1546-43a1-b61d-01f8eefe3c82-snapshots\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:33.080490 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.080329    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e93e581e-424f-4960-b2e9-fa94c29f0779-installation-pull-secrets\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.080490 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.080306    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e93e581e-424f-4960-b2e9-fa94c29f0779-ca-trust-extracted\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.080490 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.080366    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 11:25:33.080490 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.080379    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-784668b6-dtn6x: secret "image-registry-tls" not found
Sep 15 11:25:33.080490 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.080391    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2667a7e4-1546-43a1-b61d-01f8eefe3c82-trusted-ca-bundle\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:33.080712 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.080506    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls podName:e93e581e-424f-4960-b2e9-fa94c29f0779 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:33.580488154 +0000 UTC m=+35.073265286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls") pod "image-registry-784668b6-dtn6x" (UID: "e93e581e-424f-4960-b2e9-fa94c29f0779") : secret "image-registry-tls" not found
Sep 15 11:25:33.080889 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.080522    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-n7sgg\" (UID: \"e1520a65-2dce-41ce-be10-94731499c0bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"
Sep 15 11:25:33.080966 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.080918    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-default-certificate\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:33.081019 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.080962    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2667a7e4-1546-43a1-b61d-01f8eefe3c82-service-ca-bundle\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:33.081019 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.080996    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-bound-sa-token\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.081151 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.081029    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfrbh\" (UniqueName: \"kubernetes.io/projected/6cdd8d68-d249-46a5-9eed-37d9cb155861-kube-api-access-sfrbh\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:33.081151 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.081057    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c8329b83-30a0-4ead-9c95-57f0916610aa-serving-cert\") pod \"service-ca-operator-cf7c776b7-n5zv5\" (UID: \"c8329b83-30a0-4ead-9c95-57f0916610aa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5"
Sep 15 11:25:33.081151 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.081091    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2667a7e4-1546-43a1-b61d-01f8eefe3c82-tmp\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:33.081151 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.081121    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-certificates\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.081341 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.081159    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21196aa4-a811-43f9-b006-1f448fce3ea7-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7zt9z\" (UID: \"21196aa4-a811-43f9-b006-1f448fce3ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z"
Sep 15 11:25:33.081341 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.081194    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2667a7e4-1546-43a1-b61d-01f8eefe3c82-serving-cert\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:33.081341 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.081219    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:33.081341 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.081248    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-stats-auth\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:33.081341 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.081272    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e93e581e-424f-4960-b2e9-fa94c29f0779-trusted-ca\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.081603 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.081484    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2667a7e4-1546-43a1-b61d-01f8eefe3c82-trusted-ca-bundle\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:33.081603 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.081502    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2667a7e4-1546-43a1-b61d-01f8eefe3c82-snapshots\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:33.081709 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.081612    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2667a7e4-1546-43a1-b61d-01f8eefe3c82-tmp\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:33.081823 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.081773    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 11:25:33.081888 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.081868    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls podName:e1520a65-2dce-41ce-be10-94731499c0bd nodeName:}" failed. No retries permitted until 2026-09-15 11:25:33.58184061 +0000 UTC m=+35.074617749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-n7sgg" (UID: "e1520a65-2dce-41ce-be10-94731499c0bd") : secret "samples-operator-tls" not found
Sep 15 11:25:33.082194 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.081976    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21196aa4-a811-43f9-b006-1f448fce3ea7-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7zt9z\" (UID: \"21196aa4-a811-43f9-b006-1f448fce3ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z"
Sep 15 11:25:33.082194 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.082052    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c8329b83-30a0-4ead-9c95-57f0916610aa-config\") pod \"service-ca-operator-cf7c776b7-n5zv5\" (UID: \"c8329b83-30a0-4ead-9c95-57f0916610aa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5"
Sep 15 11:25:33.082194 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.082057    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2667a7e4-1546-43a1-b61d-01f8eefe3c82-service-ca-bundle\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:33.082194 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.082149    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sc2v\" (UniqueName: \"kubernetes.io/projected/2667a7e4-1546-43a1-b61d-01f8eefe3c82-kube-api-access-4sc2v\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:33.082417 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.082194    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z52f\" (UniqueName: \"kubernetes.io/projected/21196aa4-a811-43f9-b006-1f448fce3ea7-kube-api-access-6z52f\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7zt9z\" (UID: \"21196aa4-a811-43f9-b006-1f448fce3ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z"
Sep 15 11:25:33.082417 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.082294    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-certificates\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.084311 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.084289    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z"]
Sep 15 11:25:33.084402 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.084338    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-784668b6-dtn6x"]
Sep 15 11:25:33.084402 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.084358    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"]
Sep 15 11:25:33.084402 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.084389    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-6rnxm"]
Sep 15 11:25:33.084541 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.084414    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"]
Sep 15 11:25:33.084541 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.084433    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5"]
Sep 15 11:25:33.084541 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.084457    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-njsmc"]
Sep 15 11:25:33.084541 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.084475    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-568d798994-hm7cl"]
Sep 15 11:25:33.084541 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.084501    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-prrz4"]
Sep 15 11:25:33.085407 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.085252    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:33.085509 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.085488    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2667a7e4-1546-43a1-b61d-01f8eefe3c82-serving-cert\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:33.087816 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.087797    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e93e581e-424f-4960-b2e9-fa94c29f0779-installation-pull-secrets\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.087914 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.087890    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 15 11:25:33.087914 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.087908    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 15 11:25:33.088027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.087934    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-55wz6\""
Sep 15 11:25:33.088988 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.088969    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e93e581e-424f-4960-b2e9-fa94c29f0779-image-registry-private-configuration\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.092616 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.092593    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvzbf\" (UniqueName: \"kubernetes.io/projected/e1520a65-2dce-41ce-be10-94731499c0bd-kube-api-access-rvzbf\") pod \"cluster-samples-operator-86499c89c4-n7sgg\" (UID: \"e1520a65-2dce-41ce-be10-94731499c0bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"
Sep 15 11:25:33.092904 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.092880    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66vdv\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-kube-api-access-66vdv\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.093126 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.093109    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-bound-sa-token\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.093751 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.093732    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sc2v\" (UniqueName: \"kubernetes.io/projected/2667a7e4-1546-43a1-b61d-01f8eefe3c82-kube-api-access-4sc2v\") pod \"insights-operator-5844b856db-njsmc\" (UID: \"2667a7e4-1546-43a1-b61d-01f8eefe3c82\") " pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:33.114835 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.114817    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c4gb7"]
Sep 15 11:25:33.114914 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.114846    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-prrz4"]
Sep 15 11:25:33.114914 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.114858    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-dw8hm"]
Sep 15 11:25:33.114981 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.114915    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-prrz4"
Sep 15 11:25:33.117253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.117231    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 15 11:25:33.117316 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.117294    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 15 11:25:33.117518 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.117503    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fh4vk\""
Sep 15 11:25:33.117576 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.117532    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 15 11:25:33.182525 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.182498    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mthxb\" (UniqueName: \"kubernetes.io/projected/46058d0a-ac82-4cd3-9482-faf534e13869-kube-api-access-mthxb\") pod \"console-operator-696884fb9c-dw8hm\" (UID: \"46058d0a-ac82-4cd3-9482-faf534e13869\") " pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:33.182525 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.182532    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvxf2\" (UniqueName: \"kubernetes.io/projected/c8329b83-30a0-4ead-9c95-57f0916610aa-kube-api-access-jvxf2\") pod \"service-ca-operator-cf7c776b7-n5zv5\" (UID: \"c8329b83-30a0-4ead-9c95-57f0916610aa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5"
Sep 15 11:25:33.182815 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.182580    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7q7fj\" (UID: \"34006888-4d27-47db-a314-26878e17bdff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:25:33.182815 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.182638    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46058d0a-ac82-4cd3-9482-faf534e13869-serving-cert\") pod \"console-operator-696884fb9c-dw8hm\" (UID: \"46058d0a-ac82-4cd3-9482-faf534e13869\") " pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:33.182815 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.182688    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21196aa4-a811-43f9-b006-1f448fce3ea7-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7zt9z\" (UID: \"21196aa4-a811-43f9-b006-1f448fce3ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z"
Sep 15 11:25:33.182815 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.182731    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfrbh\" (UniqueName: \"kubernetes.io/projected/6cdd8d68-d249-46a5-9eed-37d9cb155861-kube-api-access-sfrbh\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:33.182815 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.182754    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c8329b83-30a0-4ead-9c95-57f0916610aa-serving-cert\") pod \"service-ca-operator-cf7c776b7-n5zv5\" (UID: \"c8329b83-30a0-4ead-9c95-57f0916610aa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5"
Sep 15 11:25:33.182815 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.182777    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21196aa4-a811-43f9-b006-1f448fce3ea7-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7zt9z\" (UID: \"21196aa4-a811-43f9-b006-1f448fce3ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z"
Sep 15 11:25:33.182815 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.182804    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/34006888-4d27-47db-a314-26878e17bdff-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-7q7fj\" (UID: \"34006888-4d27-47db-a314-26878e17bdff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:25:33.183158 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.182830    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46058d0a-ac82-4cd3-9482-faf534e13869-trusted-ca\") pod \"console-operator-696884fb9c-dw8hm\" (UID: \"46058d0a-ac82-4cd3-9482-faf534e13869\") " pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:33.183158 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.182857    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-stats-auth\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:33.183158 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.182885    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z52f\" (UniqueName: \"kubernetes.io/projected/21196aa4-a811-43f9-b006-1f448fce3ea7-kube-api-access-6z52f\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7zt9z\" (UID: \"21196aa4-a811-43f9-b006-1f448fce3ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z"
Sep 15 11:25:33.183158 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.182924    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c8329b83-30a0-4ead-9c95-57f0916610aa-config\") pod \"service-ca-operator-cf7c776b7-n5zv5\" (UID: \"c8329b83-30a0-4ead-9c95-57f0916610aa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5"
Sep 15 11:25:33.183158 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.182966    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46058d0a-ac82-4cd3-9482-faf534e13869-config\") pod \"console-operator-696884fb9c-dw8hm\" (UID: \"46058d0a-ac82-4cd3-9482-faf534e13869\") " pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:33.183158 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.182994    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nvm6\" (UniqueName: \"kubernetes.io/projected/a98bf294-4c61-4611-959d-e4941241a01b-kube-api-access-9nvm6\") pod \"volume-data-source-validator-86695b7cc7-6rnxm\" (UID: \"a98bf294-4c61-4611-959d-e4941241a01b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-6rnxm"
Sep 15 11:25:33.183158 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.183020    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spmjl\" (UniqueName: \"kubernetes.io/projected/34006888-4d27-47db-a314-26878e17bdff-kube-api-access-spmjl\") pod \"cluster-monitoring-operator-6cd954b897-7q7fj\" (UID: \"34006888-4d27-47db-a314-26878e17bdff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:25:33.183158 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.183046    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:33.183158 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.183077    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-default-certificate\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:33.183158 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.183103    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:33.183158 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.183147    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:33.183710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.183197    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np7sw\" (UniqueName: \"kubernetes.io/projected/9c09db0b-a1bd-4fe6-9009-567294abff0f-kube-api-access-np7sw\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:33.183710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.183248    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c09db0b-a1bd-4fe6-9009-567294abff0f-tmp-dir\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:33.183710 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.183535    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle podName:6cdd8d68-d249-46a5-9eed-37d9cb155861 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:33.683513131 +0000 UTC m=+35.176290274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle") pod "router-default-568d798994-hm7cl" (UID: "6cdd8d68-d249-46a5-9eed-37d9cb155861") : configmap references non-existent config key: service-ca.crt
Sep 15 11:25:33.183710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.183632    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21196aa4-a811-43f9-b006-1f448fce3ea7-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7zt9z\" (UID: \"21196aa4-a811-43f9-b006-1f448fce3ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z"
Sep 15 11:25:33.183911 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.183727    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9c09db0b-a1bd-4fe6-9009-567294abff0f-config-volume\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:33.183911 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.183804    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 11:25:33.183911 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.183820    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c8329b83-30a0-4ead-9c95-57f0916610aa-config\") pod \"service-ca-operator-cf7c776b7-n5zv5\" (UID: \"c8329b83-30a0-4ead-9c95-57f0916610aa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5"
Sep 15 11:25:33.183911 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.183871    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs podName:6cdd8d68-d249-46a5-9eed-37d9cb155861 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:33.683854136 +0000 UTC m=+35.176631265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs") pod "router-default-568d798994-hm7cl" (UID: "6cdd8d68-d249-46a5-9eed-37d9cb155861") : secret "router-metrics-certs-default" not found
Sep 15 11:25:33.186581 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.186326    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21196aa4-a811-43f9-b006-1f448fce3ea7-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7zt9z\" (UID: \"21196aa4-a811-43f9-b006-1f448fce3ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z"
Sep 15 11:25:33.186581 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.186449    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c8329b83-30a0-4ead-9c95-57f0916610aa-serving-cert\") pod \"service-ca-operator-cf7c776b7-n5zv5\" (UID: \"c8329b83-30a0-4ead-9c95-57f0916610aa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5"
Sep 15 11:25:33.186754 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.186467    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-default-certificate\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:33.186754 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.186494    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-stats-auth\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:33.186754 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.186719    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhdb5" event={"ID":"1bd5c541-ba2b-4ab4-a874-9c0436910724","Type":"ContainerStarted","Data":"bca09600f6e3ddac20807f955637d74441ee93d1b54d4ff9b4ab876dbcb3edfb"}
Sep 15 11:25:33.196357 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.196329    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z52f\" (UniqueName: \"kubernetes.io/projected/21196aa4-a811-43f9-b006-1f448fce3ea7-kube-api-access-6z52f\") pod \"kube-storage-version-migrator-operator-6c8f84f897-7zt9z\" (UID: \"21196aa4-a811-43f9-b006-1f448fce3ea7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z"
Sep 15 11:25:33.196822 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.196800    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvxf2\" (UniqueName: \"kubernetes.io/projected/c8329b83-30a0-4ead-9c95-57f0916610aa-kube-api-access-jvxf2\") pod \"service-ca-operator-cf7c776b7-n5zv5\" (UID: \"c8329b83-30a0-4ead-9c95-57f0916610aa\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5"
Sep 15 11:25:33.198020 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.198002    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfrbh\" (UniqueName: \"kubernetes.io/projected/6cdd8d68-d249-46a5-9eed-37d9cb155861-kube-api-access-sfrbh\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:33.198363 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.198345    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nvm6\" (UniqueName: \"kubernetes.io/projected/a98bf294-4c61-4611-959d-e4941241a01b-kube-api-access-9nvm6\") pod \"volume-data-source-validator-86695b7cc7-6rnxm\" (UID: \"a98bf294-4c61-4611-959d-e4941241a01b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-6rnxm"
Sep 15 11:25:33.204191 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.204167    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-njsmc"
Sep 15 11:25:33.245124 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.245096    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z"
Sep 15 11:25:33.281158 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.281135    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-6rnxm"
Sep 15 11:25:33.284501 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.284477    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np7sw\" (UniqueName: \"kubernetes.io/projected/9c09db0b-a1bd-4fe6-9009-567294abff0f-kube-api-access-np7sw\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:33.284611 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.284510    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c09db0b-a1bd-4fe6-9009-567294abff0f-tmp-dir\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:33.284611 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.284537    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9c09db0b-a1bd-4fe6-9009-567294abff0f-config-volume\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:33.284611 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.284579    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mthxb\" (UniqueName: \"kubernetes.io/projected/46058d0a-ac82-4cd3-9482-faf534e13869-kube-api-access-mthxb\") pod \"console-operator-696884fb9c-dw8hm\" (UID: \"46058d0a-ac82-4cd3-9482-faf534e13869\") " pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:33.284772 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.284618    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7q7fj\" (UID: \"34006888-4d27-47db-a314-26878e17bdff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:25:33.284772 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.284642    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46058d0a-ac82-4cd3-9482-faf534e13869-serving-cert\") pod \"console-operator-696884fb9c-dw8hm\" (UID: \"46058d0a-ac82-4cd3-9482-faf534e13869\") " pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:33.284772 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.284703    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/34006888-4d27-47db-a314-26878e17bdff-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-7q7fj\" (UID: \"34006888-4d27-47db-a314-26878e17bdff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:25:33.284772 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.284726    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46058d0a-ac82-4cd3-9482-faf534e13869-trusted-ca\") pod \"console-operator-696884fb9c-dw8hm\" (UID: \"46058d0a-ac82-4cd3-9482-faf534e13869\") " pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:33.284990 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.284774    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmfrs\" (UniqueName: \"kubernetes.io/projected/9067bd21-a937-4395-88e4-327269b84de3-kube-api-access-dmfrs\") pod \"ingress-canary-prrz4\" (UID: \"9067bd21-a937-4395-88e4-327269b84de3\") " pod="openshift-ingress-canary/ingress-canary-prrz4"
Sep 15 11:25:33.284990 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.284812    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46058d0a-ac82-4cd3-9482-faf534e13869-config\") pod \"console-operator-696884fb9c-dw8hm\" (UID: \"46058d0a-ac82-4cd3-9482-faf534e13869\") " pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:33.284990 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.284841    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spmjl\" (UniqueName: \"kubernetes.io/projected/34006888-4d27-47db-a314-26878e17bdff-kube-api-access-spmjl\") pod \"cluster-monitoring-operator-6cd954b897-7q7fj\" (UID: \"34006888-4d27-47db-a314-26878e17bdff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:25:33.284990 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.284867    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9067bd21-a937-4395-88e4-327269b84de3-cert\") pod \"ingress-canary-prrz4\" (UID: \"9067bd21-a937-4395-88e4-327269b84de3\") " pod="openshift-ingress-canary/ingress-canary-prrz4"
Sep 15 11:25:33.284990 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.284879    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c09db0b-a1bd-4fe6-9009-567294abff0f-tmp-dir\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:33.284990 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.284951    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:33.284990 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.284973    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 11:25:33.285296 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.285126    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9c09db0b-a1bd-4fe6-9009-567294abff0f-config-volume\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:33.285296 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.285192    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 11:25:33.285296 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.285247    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls podName:9c09db0b-a1bd-4fe6-9009-567294abff0f nodeName:}" failed. No retries permitted until 2026-09-15 11:25:33.785227655 +0000 UTC m=+35.278004797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls") pod "dns-default-c4gb7" (UID: "9c09db0b-a1bd-4fe6-9009-567294abff0f") : secret "dns-default-metrics-tls" not found
Sep 15 11:25:33.285540 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.285517    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls podName:34006888-4d27-47db-a314-26878e17bdff nodeName:}" failed. No retries permitted until 2026-09-15 11:25:33.785498686 +0000 UTC m=+35.278275826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-7q7fj" (UID: "34006888-4d27-47db-a314-26878e17bdff") : secret "cluster-monitoring-operator-tls" not found
Sep 15 11:25:33.285827 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.285806    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46058d0a-ac82-4cd3-9482-faf534e13869-config\") pod \"console-operator-696884fb9c-dw8hm\" (UID: \"46058d0a-ac82-4cd3-9482-faf534e13869\") " pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:33.286024 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.286007    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/46058d0a-ac82-4cd3-9482-faf534e13869-trusted-ca\") pod \"console-operator-696884fb9c-dw8hm\" (UID: \"46058d0a-ac82-4cd3-9482-faf534e13869\") " pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:33.286132 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.286111    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/34006888-4d27-47db-a314-26878e17bdff-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-7q7fj\" (UID: \"34006888-4d27-47db-a314-26878e17bdff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:25:33.287758 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.287736    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46058d0a-ac82-4cd3-9482-faf534e13869-serving-cert\") pod \"console-operator-696884fb9c-dw8hm\" (UID: \"46058d0a-ac82-4cd3-9482-faf534e13869\") " pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:33.293026 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.293004    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np7sw\" (UniqueName: \"kubernetes.io/projected/9c09db0b-a1bd-4fe6-9009-567294abff0f-kube-api-access-np7sw\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:33.293092 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.293023    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mthxb\" (UniqueName: \"kubernetes.io/projected/46058d0a-ac82-4cd3-9482-faf534e13869-kube-api-access-mthxb\") pod \"console-operator-696884fb9c-dw8hm\" (UID: \"46058d0a-ac82-4cd3-9482-faf534e13869\") " pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:33.293092 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.293070    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spmjl\" (UniqueName: \"kubernetes.io/projected/34006888-4d27-47db-a314-26878e17bdff-kube-api-access-spmjl\") pod \"cluster-monitoring-operator-6cd954b897-7q7fj\" (UID: \"34006888-4d27-47db-a314-26878e17bdff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:25:33.301391 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.301378    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5"
Sep 15 11:25:33.334327 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.334307    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:33.385840 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.385817    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmfrs\" (UniqueName: \"kubernetes.io/projected/9067bd21-a937-4395-88e4-327269b84de3-kube-api-access-dmfrs\") pod \"ingress-canary-prrz4\" (UID: \"9067bd21-a937-4395-88e4-327269b84de3\") " pod="openshift-ingress-canary/ingress-canary-prrz4"
Sep 15 11:25:33.385895 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.385853    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9067bd21-a937-4395-88e4-327269b84de3-cert\") pod \"ingress-canary-prrz4\" (UID: \"9067bd21-a937-4395-88e4-327269b84de3\") " pod="openshift-ingress-canary/ingress-canary-prrz4"
Sep 15 11:25:33.387905 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.387888    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9067bd21-a937-4395-88e4-327269b84de3-cert\") pod \"ingress-canary-prrz4\" (UID: \"9067bd21-a937-4395-88e4-327269b84de3\") " pod="openshift-ingress-canary/ingress-canary-prrz4"
Sep 15 11:25:33.393244 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.393228    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmfrs\" (UniqueName: \"kubernetes.io/projected/9067bd21-a937-4395-88e4-327269b84de3-kube-api-access-dmfrs\") pod \"ingress-canary-prrz4\" (UID: \"9067bd21-a937-4395-88e4-327269b84de3\") " pod="openshift-ingress-canary/ingress-canary-prrz4"
Sep 15 11:25:33.422415 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.422395    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-prrz4"
Sep 15 11:25:33.533544 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.533521    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-njsmc"]
Sep 15 11:25:33.536920 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:33.536865    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2667a7e4_1546_43a1_b61d_01f8eefe3c82.slice/crio-ba7f61651b148b3f93c0246f40bc8c736a01ac32714a46afb4ba83179c1a5128 WatchSource:0}: Error finding container ba7f61651b148b3f93c0246f40bc8c736a01ac32714a46afb4ba83179c1a5128: Status 404 returned error can't find the container with id ba7f61651b148b3f93c0246f40bc8c736a01ac32714a46afb4ba83179c1a5128
Sep 15 11:25:33.587452 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.587409    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:33.587588 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.587550    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-n7sgg\" (UID: \"e1520a65-2dce-41ce-be10-94731499c0bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"
Sep 15 11:25:33.587776 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.587734    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 11:25:33.587835 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.587801    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls podName:e1520a65-2dce-41ce-be10-94731499c0bd nodeName:}" failed. No retries permitted until 2026-09-15 11:25:34.587782622 +0000 UTC m=+36.080559750 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-n7sgg" (UID: "e1520a65-2dce-41ce-be10-94731499c0bd") : secret "samples-operator-tls" not found
Sep 15 11:25:33.587982 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.587960    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 11:25:33.588060 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.587983    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-784668b6-dtn6x: secret "image-registry-tls" not found
Sep 15 11:25:33.588060 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.588043    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls podName:e93e581e-424f-4960-b2e9-fa94c29f0779 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:34.588024458 +0000 UTC m=+36.080801605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls") pod "image-registry-784668b6-dtn6x" (UID: "e93e581e-424f-4960-b2e9-fa94c29f0779") : secret "image-registry-tls" not found
Sep 15 11:25:33.688240 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.688178    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:33.688240 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.688229    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:33.688440 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.688334    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle podName:6cdd8d68-d249-46a5-9eed-37d9cb155861 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:34.688321379 +0000 UTC m=+36.181098504 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle") pod "router-default-568d798994-hm7cl" (UID: "6cdd8d68-d249-46a5-9eed-37d9cb155861") : configmap references non-existent config key: service-ca.crt
Sep 15 11:25:33.688440 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.688345    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 11:25:33.688440 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.688408    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs podName:6cdd8d68-d249-46a5-9eed-37d9cb155861 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:34.688389251 +0000 UTC m=+36.181166398 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs") pod "router-default-568d798994-hm7cl" (UID: "6cdd8d68-d249-46a5-9eed-37d9cb155861") : secret "router-metrics-certs-default" not found
Sep 15 11:25:33.807349 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.789916    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:33.807349 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.790035    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7q7fj\" (UID: \"34006888-4d27-47db-a314-26878e17bdff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:25:33.807349 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.790230    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 11:25:33.807349 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.790304    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls podName:34006888-4d27-47db-a314-26878e17bdff nodeName:}" failed. No retries permitted until 2026-09-15 11:25:34.790268919 +0000 UTC m=+36.283046059 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-7q7fj" (UID: "34006888-4d27-47db-a314-26878e17bdff") : secret "cluster-monitoring-operator-tls" not found
Sep 15 11:25:33.807349 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.790764    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 11:25:33.807349 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:33.790830    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls podName:9c09db0b-a1bd-4fe6-9009-567294abff0f nodeName:}" failed. No retries permitted until 2026-09-15 11:25:34.790798858 +0000 UTC m=+36.283576000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls") pod "dns-default-c4gb7" (UID: "9c09db0b-a1bd-4fe6-9009-567294abff0f") : secret "dns-default-metrics-tls" not found
Sep 15 11:25:33.815070 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.815039    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z"]
Sep 15 11:25:33.820350 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:33.820327    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21196aa4_a811_43f9_b006_1f448fce3ea7.slice/crio-d8f23595fddd362dcd18bfd9aad33e9bc7520300f694f394f5d94330bf3370e9 WatchSource:0}: Error finding container d8f23595fddd362dcd18bfd9aad33e9bc7520300f694f394f5d94330bf3370e9: Status 404 returned error can't find the container with id d8f23595fddd362dcd18bfd9aad33e9bc7520300f694f394f5d94330bf3370e9
Sep 15 11:25:33.828930 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.828906    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-prrz4"]
Sep 15 11:25:33.832306 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:33.832260    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9067bd21_a937_4395_88e4_327269b84de3.slice/crio-a7f5323459d4c32eb790092f952a68950088b328c3697cd673850e11939e7960 WatchSource:0}: Error finding container a7f5323459d4c32eb790092f952a68950088b328c3697cd673850e11939e7960: Status 404 returned error can't find the container with id a7f5323459d4c32eb790092f952a68950088b328c3697cd673850e11939e7960
Sep 15 11:25:33.832643 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.832597    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-6rnxm"]
Sep 15 11:25:33.833731 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.833702    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-dw8hm"]
Sep 15 11:25:33.834934 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:33.834879    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5"]
Sep 15 11:25:33.836081 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:33.836045    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8329b83_30a0_4ead_9c95_57f0916610aa.slice/crio-993be8e884a5c1e9d3b5acd27edb11b72c666588e4bb2487b70da88da62b6fbc WatchSource:0}: Error finding container 993be8e884a5c1e9d3b5acd27edb11b72c666588e4bb2487b70da88da62b6fbc: Status 404 returned error can't find the container with id 993be8e884a5c1e9d3b5acd27edb11b72c666588e4bb2487b70da88da62b6fbc
Sep 15 11:25:33.836716 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:33.836699    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda98bf294_4c61_4611_959d_e4941241a01b.slice/crio-7498d072cb04547ba451cb8799cec975f38881f2ff21a6e359f638ecc77cf5de WatchSource:0}: Error finding container 7498d072cb04547ba451cb8799cec975f38881f2ff21a6e359f638ecc77cf5de: Status 404 returned error can't find the container with id 7498d072cb04547ba451cb8799cec975f38881f2ff21a6e359f638ecc77cf5de
Sep 15 11:25:33.837481 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:33.837414    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46058d0a_ac82_4cd3_9482_faf534e13869.slice/crio-5c7f59a7f91a1c1db29a73782533a3428f9284d075c8d8b5ed5a2d124c3a4aef WatchSource:0}: Error finding container 5c7f59a7f91a1c1db29a73782533a3428f9284d075c8d8b5ed5a2d124c3a4aef: Status 404 returned error can't find the container with id 5c7f59a7f91a1c1db29a73782533a3428f9284d075c8d8b5ed5a2d124c3a4aef
Sep 15 11:25:34.028902 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.028827    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:25:34.029030 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.028918    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:25:34.029030 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.028947    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:25:34.031226 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.031195    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4fvsz\""
Sep 15 11:25:34.031226 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.031210    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 11:25:34.031963 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.031192    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 11:25:34.031963 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.031287    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n59hg\""
Sep 15 11:25:34.031963 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.031292    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 11:25:34.031963 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.031724    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 11:25:34.194034 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.194002    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-njsmc" event={"ID":"2667a7e4-1546-43a1-b61d-01f8eefe3c82","Type":"ContainerStarted","Data":"ba7f61651b148b3f93c0246f40bc8c736a01ac32714a46afb4ba83179c1a5128"}
Sep 15 11:25:34.195170 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.195140    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-6rnxm" event={"ID":"a98bf294-4c61-4611-959d-e4941241a01b","Type":"ContainerStarted","Data":"7498d072cb04547ba451cb8799cec975f38881f2ff21a6e359f638ecc77cf5de"}
Sep 15 11:25:34.197815 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.197793    2560 generic.go:356] "Generic (PLEG): container finished" podID="1bd5c541-ba2b-4ab4-a874-9c0436910724" containerID="bca09600f6e3ddac20807f955637d74441ee93d1b54d4ff9b4ab876dbcb3edfb" exitCode=0
Sep 15 11:25:34.197919 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.197832    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhdb5" event={"ID":"1bd5c541-ba2b-4ab4-a874-9c0436910724","Type":"ContainerDied","Data":"bca09600f6e3ddac20807f955637d74441ee93d1b54d4ff9b4ab876dbcb3edfb"}
Sep 15 11:25:34.198970 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.198951    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-dw8hm" event={"ID":"46058d0a-ac82-4cd3-9482-faf534e13869","Type":"ContainerStarted","Data":"5c7f59a7f91a1c1db29a73782533a3428f9284d075c8d8b5ed5a2d124c3a4aef"}
Sep 15 11:25:34.200166 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.200140    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5" event={"ID":"c8329b83-30a0-4ead-9c95-57f0916610aa","Type":"ContainerStarted","Data":"993be8e884a5c1e9d3b5acd27edb11b72c666588e4bb2487b70da88da62b6fbc"}
Sep 15 11:25:34.201472 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.201454    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-prrz4" event={"ID":"9067bd21-a937-4395-88e4-327269b84de3","Type":"ContainerStarted","Data":"a7f5323459d4c32eb790092f952a68950088b328c3697cd673850e11939e7960"}
Sep 15 11:25:34.202580 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.202549    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z" event={"ID":"21196aa4-a811-43f9-b006-1f448fce3ea7","Type":"ContainerStarted","Data":"d8f23595fddd362dcd18bfd9aad33e9bc7520300f694f394f5d94330bf3370e9"}
Sep 15 11:25:34.597163 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.597123    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:34.597353 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.597278    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-n7sgg\" (UID: \"e1520a65-2dce-41ce-be10-94731499c0bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"
Sep 15 11:25:34.597485 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:34.597468    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 11:25:34.597557 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:34.597537    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls podName:e1520a65-2dce-41ce-be10-94731499c0bd nodeName:}" failed. No retries permitted until 2026-09-15 11:25:36.597517847 +0000 UTC m=+38.090294978 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-n7sgg" (UID: "e1520a65-2dce-41ce-be10-94731499c0bd") : secret "samples-operator-tls" not found
Sep 15 11:25:34.597656 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:34.597604    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 11:25:34.597656 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:34.597618    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-784668b6-dtn6x: secret "image-registry-tls" not found
Sep 15 11:25:34.597808 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:34.597671    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls podName:e93e581e-424f-4960-b2e9-fa94c29f0779 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:36.597657274 +0000 UTC m=+38.090434412 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls") pod "image-registry-784668b6-dtn6x" (UID: "e93e581e-424f-4960-b2e9-fa94c29f0779") : secret "image-registry-tls" not found
Sep 15 11:25:34.698679 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.698641    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:34.698845 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.698723    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:34.698997 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:34.698978    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle podName:6cdd8d68-d249-46a5-9eed-37d9cb155861 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:36.698957972 +0000 UTC m=+38.191735111 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle") pod "router-default-568d798994-hm7cl" (UID: "6cdd8d68-d249-46a5-9eed-37d9cb155861") : configmap references non-existent config key: service-ca.crt
Sep 15 11:25:34.699429 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:34.699404    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 11:25:34.699542 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:34.699465    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs podName:6cdd8d68-d249-46a5-9eed-37d9cb155861 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:36.699449629 +0000 UTC m=+38.192226757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs") pod "router-default-568d798994-hm7cl" (UID: "6cdd8d68-d249-46a5-9eed-37d9cb155861") : secret "router-metrics-certs-default" not found
Sep 15 11:25:34.799452 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.799420    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7q7fj\" (UID: \"34006888-4d27-47db-a314-26878e17bdff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:25:34.799638 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:34.799548    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:34.799713 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:34.799696    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 11:25:34.799774 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:34.799754    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls podName:9c09db0b-a1bd-4fe6-9009-567294abff0f nodeName:}" failed. No retries permitted until 2026-09-15 11:25:36.79973562 +0000 UTC m=+38.292512748 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls") pod "dns-default-c4gb7" (UID: "9c09db0b-a1bd-4fe6-9009-567294abff0f") : secret "dns-default-metrics-tls" not found
Sep 15 11:25:34.800132 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:34.800110    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 11:25:34.800235 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:34.800171    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls podName:34006888-4d27-47db-a314-26878e17bdff nodeName:}" failed. No retries permitted until 2026-09-15 11:25:36.800154954 +0000 UTC m=+38.292932099 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-7q7fj" (UID: "34006888-4d27-47db-a314-26878e17bdff") : secret "cluster-monitoring-operator-tls" not found
Sep 15 11:25:35.214647 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:35.214006    2560 generic.go:356] "Generic (PLEG): container finished" podID="1bd5c541-ba2b-4ab4-a874-9c0436910724" containerID="4c8327cf01437d73362b6ba7732786d4336113ee9584e652a33bd30410763d39" exitCode=0
Sep 15 11:25:35.214647 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:35.214121    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhdb5" event={"ID":"1bd5c541-ba2b-4ab4-a874-9c0436910724","Type":"ContainerDied","Data":"4c8327cf01437d73362b6ba7732786d4336113ee9584e652a33bd30410763d39"}
Sep 15 11:25:36.221696 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:36.221656    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lhdb5" event={"ID":"1bd5c541-ba2b-4ab4-a874-9c0436910724","Type":"ContainerStarted","Data":"78be9e2ebb3fd9178ecd34512adc502e031ce6ce65685c26b1ff2d148a373104"}
Sep 15 11:25:36.242793 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:36.242724    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lhdb5" podStartSLOduration=7.884877072 podStartE2EDuration="37.242702081s" totalImagesPullingTime="29.357825009s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 11:24:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:36.241671136 +0000 UTC m=+37.734448284" watchObservedRunningTime="2026-09-15 11:25:36.242702081 +0000 UTC m=+37.735479229"
Sep 15 11:25:36.385860 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:36.385825    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:36.471471 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:36.471429    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:36.619172 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:36.619134    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-n7sgg\" (UID: \"e1520a65-2dce-41ce-be10-94731499c0bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"
Sep 15 11:25:36.619342 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:36.619243    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:36.619342 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:36.619284    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 11:25:36.619342 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:36.619337    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 11:25:36.619513 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:36.619348    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-784668b6-dtn6x: secret "image-registry-tls" not found
Sep 15 11:25:36.619513 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:36.619353    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls podName:e1520a65-2dce-41ce-be10-94731499c0bd nodeName:}" failed. No retries permitted until 2026-09-15 11:25:40.619332405 +0000 UTC m=+42.112109537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-n7sgg" (UID: "e1520a65-2dce-41ce-be10-94731499c0bd") : secret "samples-operator-tls" not found
Sep 15 11:25:36.619513 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:36.619381    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls podName:e93e581e-424f-4960-b2e9-fa94c29f0779 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:40.619372231 +0000 UTC m=+42.112149368 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls") pod "image-registry-784668b6-dtn6x" (UID: "e93e581e-424f-4960-b2e9-fa94c29f0779") : secret "image-registry-tls" not found
Sep 15 11:25:36.719967 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:36.719931    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:36.720134 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:36.720002    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:36.720134 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:36.720117    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 11:25:36.720272 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:36.720134    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle podName:6cdd8d68-d249-46a5-9eed-37d9cb155861 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:40.720115253 +0000 UTC m=+42.212892383 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle") pod "router-default-568d798994-hm7cl" (UID: "6cdd8d68-d249-46a5-9eed-37d9cb155861") : configmap references non-existent config key: service-ca.crt
Sep 15 11:25:36.720272 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:36.720175    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs podName:6cdd8d68-d249-46a5-9eed-37d9cb155861 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:40.720161659 +0000 UTC m=+42.212938785 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs") pod "router-default-568d798994-hm7cl" (UID: "6cdd8d68-d249-46a5-9eed-37d9cb155861") : secret "router-metrics-certs-default" not found
Sep 15 11:25:36.821108 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:36.821036    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:36.821242 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:36.821116    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7q7fj\" (UID: \"34006888-4d27-47db-a314-26878e17bdff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:25:36.821242 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:36.821195    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 11:25:36.821242 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:36.821207    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 11:25:36.821356 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:36.821258    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls podName:9c09db0b-a1bd-4fe6-9009-567294abff0f nodeName:}" failed. No retries permitted until 2026-09-15 11:25:40.821242741 +0000 UTC m=+42.314019873 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls") pod "dns-default-c4gb7" (UID: "9c09db0b-a1bd-4fe6-9009-567294abff0f") : secret "dns-default-metrics-tls" not found
Sep 15 11:25:36.821356 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:36.821272    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls podName:34006888-4d27-47db-a314-26878e17bdff nodeName:}" failed. No retries permitted until 2026-09-15 11:25:40.821266319 +0000 UTC m=+42.314043445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-7q7fj" (UID: "34006888-4d27-47db-a314-26878e17bdff") : secret "cluster-monitoring-operator-tls" not found
Sep 15 11:25:37.770264 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:37.770228    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:37.831215 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:37.831174    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:40.235088 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.235058    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/0.log"
Sep 15 11:25:40.235488 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.235107    2560 generic.go:356] "Generic (PLEG): container finished" podID="46058d0a-ac82-4cd3-9482-faf534e13869" containerID="684d5a0679d54418fad206fc14ae4cba17d09ef79d938694789f355e255e61ee" exitCode=255
Sep 15 11:25:40.235488 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.235189    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-dw8hm" event={"ID":"46058d0a-ac82-4cd3-9482-faf534e13869","Type":"ContainerDied","Data":"684d5a0679d54418fad206fc14ae4cba17d09ef79d938694789f355e255e61ee"}
Sep 15 11:25:40.235488 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.235417    2560 scope.go:117] "RemoveContainer" containerID="684d5a0679d54418fad206fc14ae4cba17d09ef79d938694789f355e255e61ee"
Sep 15 11:25:40.237068 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.237045    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5" event={"ID":"c8329b83-30a0-4ead-9c95-57f0916610aa","Type":"ContainerStarted","Data":"96d3ca608813e6688541414c57f6103b272f866d70c115c8e93442646fe9634c"}
Sep 15 11:25:40.238847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.238819    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-prrz4" event={"ID":"9067bd21-a937-4395-88e4-327269b84de3","Type":"ContainerStarted","Data":"c9e2299fe2e73a304df87f5790dd0f90d98a54300e6feb4b4894b629f36651c2"}
Sep 15 11:25:40.240432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.240405    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z" event={"ID":"21196aa4-a811-43f9-b006-1f448fce3ea7","Type":"ContainerStarted","Data":"342d9c60210683a45d98167b7e41676d53677fe27643060ab6e30500722ec727"}
Sep 15 11:25:40.242148 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.242126    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-njsmc" event={"ID":"2667a7e4-1546-43a1-b61d-01f8eefe3c82","Type":"ContainerStarted","Data":"220ce69b91a0e4ecb7e4ea54eabf272d95893f742e0dd3c4a6c55ffe768ef266"}
Sep 15 11:25:40.243704 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.243682    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-6rnxm" event={"ID":"a98bf294-4c61-4611-959d-e4941241a01b","Type":"ContainerStarted","Data":"3e8fbdb859e8d3318468f92f01dbfc9a4b2cf85f67e804c4e80bcd85b225b361"}
Sep 15 11:25:40.266747 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.266501    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-njsmc" podStartSLOduration=12.152642381 podStartE2EDuration="18.266485923s" totalImagesPullingTime="6.113843542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:40.265958301 +0000 UTC m=+41.758735450" watchObservedRunningTime="2026-09-15 11:25:40.266485923 +0000 UTC m=+41.759263076"
Sep 15 11:25:40.285529 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.283888    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-prrz4" podStartSLOduration=2.465469163 podStartE2EDuration="8.283871652s" totalImagesPullingTime="5.818402489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:40.283030504 +0000 UTC m=+41.775807653" watchObservedRunningTime="2026-09-15 11:25:40.283871652 +0000 UTC m=+41.776648805"
Sep 15 11:25:40.310214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.310146    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z" podStartSLOduration=2.044944911 podStartE2EDuration="8.310128335s" totalImagesPullingTime="6.265183424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:40.30911097 +0000 UTC m=+41.801888118" watchObservedRunningTime="2026-09-15 11:25:40.310128335 +0000 UTC m=+41.802905477"
Sep 15 11:25:40.331104 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.331052    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-6rnxm" podStartSLOduration=3.122343099 podStartE2EDuration="8.331022168s" totalImagesPullingTime="5.208679069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:40.327993733 +0000 UTC m=+41.820770884" watchObservedRunningTime="2026-09-15 11:25:40.331022168 +0000 UTC m=+41.823799317"
Sep 15 11:25:40.347041 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.346975    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5" podStartSLOduration=2.532221653 podStartE2EDuration="8.346958698s" totalImagesPullingTime="5.814737045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:40.346737397 +0000 UTC m=+41.839514546" watchObservedRunningTime="2026-09-15 11:25:40.346958698 +0000 UTC m=+41.839735847"
Sep 15 11:25:40.654285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.654207    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:40.654447 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.654332    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-n7sgg\" (UID: \"e1520a65-2dce-41ce-be10-94731499c0bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"
Sep 15 11:25:40.654447 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:40.654388    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 11:25:40.654447 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:40.654413    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-784668b6-dtn6x: secret "image-registry-tls" not found
Sep 15 11:25:40.654447 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:40.654436    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 11:25:40.654612 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:40.654485    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls podName:e93e581e-424f-4960-b2e9-fa94c29f0779 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:48.65446252 +0000 UTC m=+50.147239652 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls") pod "image-registry-784668b6-dtn6x" (UID: "e93e581e-424f-4960-b2e9-fa94c29f0779") : secret "image-registry-tls" not found
Sep 15 11:25:40.654612 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:40.654504    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls podName:e1520a65-2dce-41ce-be10-94731499c0bd nodeName:}" failed. No retries permitted until 2026-09-15 11:25:48.654494936 +0000 UTC m=+50.147272065 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-n7sgg" (UID: "e1520a65-2dce-41ce-be10-94731499c0bd") : secret "samples-operator-tls" not found
Sep 15 11:25:40.755571 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.755528    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:40.755737 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:40.755699    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 11:25:40.755783 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:40.755773    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs podName:6cdd8d68-d249-46a5-9eed-37d9cb155861 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:48.755753483 +0000 UTC m=+50.248530613 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs") pod "router-default-568d798994-hm7cl" (UID: "6cdd8d68-d249-46a5-9eed-37d9cb155861") : secret "router-metrics-certs-default" not found
Sep 15 11:25:40.755848 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.755700    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:40.755848 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:40.755796    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle podName:6cdd8d68-d249-46a5-9eed-37d9cb155861 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:48.755781013 +0000 UTC m=+50.248558145 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle") pod "router-default-568d798994-hm7cl" (UID: "6cdd8d68-d249-46a5-9eed-37d9cb155861") : configmap references non-existent config key: service-ca.crt
Sep 15 11:25:40.857094 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.857062    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:40.857250 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:40.857145    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7q7fj\" (UID: \"34006888-4d27-47db-a314-26878e17bdff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:25:40.857250 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:40.857229    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 11:25:40.857373 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:40.857262    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 11:25:40.857373 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:40.857312    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls podName:9c09db0b-a1bd-4fe6-9009-567294abff0f nodeName:}" failed. No retries permitted until 2026-09-15 11:25:48.857289461 +0000 UTC m=+50.350066591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls") pod "dns-default-c4gb7" (UID: "9c09db0b-a1bd-4fe6-9009-567294abff0f") : secret "dns-default-metrics-tls" not found
Sep 15 11:25:40.857373 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:40.857333    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls podName:34006888-4d27-47db-a314-26878e17bdff nodeName:}" failed. No retries permitted until 2026-09-15 11:25:48.85732278 +0000 UTC m=+50.350099913 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-7q7fj" (UID: "34006888-4d27-47db-a314-26878e17bdff") : secret "cluster-monitoring-operator-tls" not found
Sep 15 11:25:41.248831 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.248799    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:25:41.249409 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.249063    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-smdnj"]
Sep 15 11:25:41.249409 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.249317    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/0.log"
Sep 15 11:25:41.249409 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.249355    2560 generic.go:356] "Generic (PLEG): container finished" podID="46058d0a-ac82-4cd3-9482-faf534e13869" containerID="417747d0f40cef8f56755b8bc5bea0904cf18f5253c1722270fef35056b036ac" exitCode=255
Sep 15 11:25:41.250389 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.250362    2560 scope.go:117] "RemoveContainer" containerID="417747d0f40cef8f56755b8bc5bea0904cf18f5253c1722270fef35056b036ac"
Sep 15 11:25:41.250584 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:41.250539    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-dw8hm_openshift-console-operator(46058d0a-ac82-4cd3-9482-faf534e13869)\"" pod="openshift-console-operator/console-operator-696884fb9c-dw8hm" podUID="46058d0a-ac82-4cd3-9482-faf534e13869"
Sep 15 11:25:41.274178 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.274149    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-smdnj"]
Sep 15 11:25:41.274317 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.274199    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-dw8hm" event={"ID":"46058d0a-ac82-4cd3-9482-faf534e13869","Type":"ContainerDied","Data":"417747d0f40cef8f56755b8bc5bea0904cf18f5253c1722270fef35056b036ac"}
Sep 15 11:25:41.274317 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.274248    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:41.274317 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.274276    2560 scope.go:117] "RemoveContainer" containerID="684d5a0679d54418fad206fc14ae4cba17d09ef79d938694789f355e255e61ee"
Sep 15 11:25:41.276832 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.276802    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 15 11:25:41.276915 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.276895    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 15 11:25:41.279316 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.279294    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8vhk8\""
Sep 15 11:25:41.463848 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.463811    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/61c49049-f9ab-48fd-8ee7-7b853de6abd1-crio-socket\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:41.464034 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.463915    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45m4d\" (UniqueName: \"kubernetes.io/projected/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-api-access-45m4d\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:41.464034 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.463951    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/61c49049-f9ab-48fd-8ee7-7b853de6abd1-data-volume\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:41.464034 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.463994    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:41.464034 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.464016    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:41.564842 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.564762    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:41.564842 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.564804    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:41.564842 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.564832    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/61c49049-f9ab-48fd-8ee7-7b853de6abd1-crio-socket\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:41.565079 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:41.564883    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 11:25:41.565079 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:41.564918    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 11:25:41.565079 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.564939    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45m4d\" (UniqueName: \"kubernetes.io/projected/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-api-access-45m4d\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:41.565079 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:41.564961    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-rbac-proxy-cm podName:61c49049-f9ab-48fd-8ee7-7b853de6abd1 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:42.064941037 +0000 UTC m=+43.557718186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-smdnj" (UID: "61c49049-f9ab-48fd-8ee7-7b853de6abd1") : configmap "kube-rbac-proxy" not found
Sep 15 11:25:41.565079 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:41.564992    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls podName:61c49049-f9ab-48fd-8ee7-7b853de6abd1 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:42.0649826 +0000 UTC m=+43.557759727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-smdnj" (UID: "61c49049-f9ab-48fd-8ee7-7b853de6abd1") : secret "insights-runtime-extractor-tls" not found
Sep 15 11:25:41.565079 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.565039    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/61c49049-f9ab-48fd-8ee7-7b853de6abd1-data-volume\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:41.565079 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.565048    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/61c49049-f9ab-48fd-8ee7-7b853de6abd1-crio-socket\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:41.565441 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.565311    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/61c49049-f9ab-48fd-8ee7-7b853de6abd1-data-volume\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:41.573661 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:41.573633    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45m4d\" (UniqueName: \"kubernetes.io/projected/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-api-access-45m4d\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:42.041062 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:42.041030    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:42.070052 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:42.070022    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:42.070193 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:42.070055    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:42.070193 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:42.070140    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 11:25:42.070193 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:42.070157    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 11:25:42.070350 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:42.070197    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-rbac-proxy-cm podName:61c49049-f9ab-48fd-8ee7-7b853de6abd1 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:43.070182468 +0000 UTC m=+44.562959607 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-smdnj" (UID: "61c49049-f9ab-48fd-8ee7-7b853de6abd1") : configmap "kube-rbac-proxy" not found
Sep 15 11:25:42.070350 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:42.070212    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls podName:61c49049-f9ab-48fd-8ee7-7b853de6abd1 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:43.070205204 +0000 UTC m=+44.562982330 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-smdnj" (UID: "61c49049-f9ab-48fd-8ee7-7b853de6abd1") : secret "insights-runtime-extractor-tls" not found
Sep 15 11:25:42.144003 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:42.143974    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:42.249851 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:42.249817    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:42.254453 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:42.254434    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:25:42.254834 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:42.254810    2560 scope.go:117] "RemoveContainer" containerID="417747d0f40cef8f56755b8bc5bea0904cf18f5253c1722270fef35056b036ac"
Sep 15 11:25:42.254997 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:42.254977    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-dw8hm_openshift-console-operator(46058d0a-ac82-4cd3-9482-faf534e13869)\"" pod="openshift-console-operator/console-operator-696884fb9c-dw8hm" podUID="46058d0a-ac82-4cd3-9482-faf534e13869"
Sep 15 11:25:42.330220 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:42.330151    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:42.393265 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:42.393234    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:43.031263 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:43.031232    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hq8r8_43830c40-9ac0-4143-b673-a73a6273075c/dns-node-resolver/0.log"
Sep 15 11:25:43.079295 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:43.079267    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:43.079445 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:43.079304    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:43.079445 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:43.079383    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 11:25:43.079445 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:43.079426    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 11:25:43.079577 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:43.079451    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-rbac-proxy-cm podName:61c49049-f9ab-48fd-8ee7-7b853de6abd1 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:45.079432559 +0000 UTC m=+46.572209701 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-smdnj" (UID: "61c49049-f9ab-48fd-8ee7-7b853de6abd1") : configmap "kube-rbac-proxy" not found
Sep 15 11:25:43.079577 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:43.079465    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls podName:61c49049-f9ab-48fd-8ee7-7b853de6abd1 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:45.079458949 +0000 UTC m=+46.572236074 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-smdnj" (UID: "61c49049-f9ab-48fd-8ee7-7b853de6abd1") : secret "insights-runtime-extractor-tls" not found
Sep 15 11:25:43.334837 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:43.334775    2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:43.334837 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:43.334822    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:43.335190 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:43.335108    2560 scope.go:117] "RemoveContainer" containerID="417747d0f40cef8f56755b8bc5bea0904cf18f5253c1722270fef35056b036ac"
Sep 15 11:25:43.335290 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:43.335272    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-dw8hm_openshift-console-operator(46058d0a-ac82-4cd3-9482-faf534e13869)\"" pod="openshift-console-operator/console-operator-696884fb9c-dw8hm" podUID="46058d0a-ac82-4cd3-9482-faf534e13869"
Sep 15 11:25:43.551166 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:43.551131    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:43.650792 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:43.650761    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:43.799316 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:43.799278    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:43.830930 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:43.830907    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xfzbq_dc0a2e63-3309-454b-890d-bc634e4ee43a/node-ca/0.log"
Sep 15 11:25:44.234329 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:44.234300    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-prrz4_9067bd21-a937-4395-88e4-327269b84de3/serve-healthcheck-canary/0.log"
Sep 15 11:25:44.613428 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:44.613394    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:45.097309 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:45.097271    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:45.097309 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:45.097314    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:45.097508 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:45.097392    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 11:25:45.097508 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:45.097413    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 11:25:45.097508 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:45.097457    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-rbac-proxy-cm podName:61c49049-f9ab-48fd-8ee7-7b853de6abd1 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:49.097439596 +0000 UTC m=+50.590216723 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-smdnj" (UID: "61c49049-f9ab-48fd-8ee7-7b853de6abd1") : configmap "kube-rbac-proxy" not found
Sep 15 11:25:45.097508 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:45.097474    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls podName:61c49049-f9ab-48fd-8ee7-7b853de6abd1 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:49.097465045 +0000 UTC m=+50.590242171 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-smdnj" (UID: "61c49049-f9ab-48fd-8ee7-7b853de6abd1") : secret "insights-runtime-extractor-tls" not found
Sep 15 11:25:47.751579 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:47.751508    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:48.727903 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:48.727852    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:25:48.728132 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:48.727975    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-n7sgg\" (UID: \"e1520a65-2dce-41ce-be10-94731499c0bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"
Sep 15 11:25:48.728132 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:48.727994    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 15 11:25:48.728132 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:48.728013    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-784668b6-dtn6x: secret "image-registry-tls" not found
Sep 15 11:25:48.728132 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:48.728066    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls podName:e93e581e-424f-4960-b2e9-fa94c29f0779 nodeName:}" failed. No retries permitted until 2026-09-15 11:26:04.728052465 +0000 UTC m=+66.220829591 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls") pod "image-registry-784668b6-dtn6x" (UID: "e93e581e-424f-4960-b2e9-fa94c29f0779") : secret "image-registry-tls" not found
Sep 15 11:25:48.728132 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:48.728066    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 15 11:25:48.728132 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:48.728094    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls podName:e1520a65-2dce-41ce-be10-94731499c0bd nodeName:}" failed. No retries permitted until 2026-09-15 11:26:04.728087936 +0000 UTC m=+66.220865083 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-n7sgg" (UID: "e1520a65-2dce-41ce-be10-94731499c0bd") : secret "samples-operator-tls" not found
Sep 15 11:25:48.765294 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:48.765257    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:48.828718 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:48.828687    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:48.828857 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:48.828832    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle podName:6cdd8d68-d249-46a5-9eed-37d9cb155861 nodeName:}" failed. No retries permitted until 2026-09-15 11:26:04.828816429 +0000 UTC m=+66.321593560 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle") pod "router-default-568d798994-hm7cl" (UID: "6cdd8d68-d249-46a5-9eed-37d9cb155861") : configmap references non-existent config key: service-ca.crt
Sep 15 11:25:48.828915 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:48.828871    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:25:48.828955 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:48.828938    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 15 11:25:48.828994 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:48.828965    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs podName:6cdd8d68-d249-46a5-9eed-37d9cb155861 nodeName:}" failed. No retries permitted until 2026-09-15 11:26:04.828957718 +0000 UTC m=+66.321734843 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs") pod "router-default-568d798994-hm7cl" (UID: "6cdd8d68-d249-46a5-9eed-37d9cb155861") : secret "router-metrics-certs-default" not found
Sep 15 11:25:48.929947 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:48.929914    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7q7fj\" (UID: \"34006888-4d27-47db-a314-26878e17bdff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:25:48.930088 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:48.930026    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:25:48.930088 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:48.930049    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 15 11:25:48.930209 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:48.930098    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls podName:34006888-4d27-47db-a314-26878e17bdff nodeName:}" failed. No retries permitted until 2026-09-15 11:26:04.93008144 +0000 UTC m=+66.422858566 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-7q7fj" (UID: "34006888-4d27-47db-a314-26878e17bdff") : secret "cluster-monitoring-operator-tls" not found
Sep 15 11:25:48.930209 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:48.930136    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 15 11:25:48.930209 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:48.930203    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls podName:9c09db0b-a1bd-4fe6-9009-567294abff0f nodeName:}" failed. No retries permitted until 2026-09-15 11:26:04.930188358 +0000 UTC m=+66.422965487 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls") pod "dns-default-c4gb7" (UID: "9c09db0b-a1bd-4fe6-9009-567294abff0f") : secret "dns-default-metrics-tls" not found
Sep 15 11:25:49.131676 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:49.131650    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:49.131816 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:49.131774    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:49.131816 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:49.131789    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 11:25:49.131885 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:25:49.131840    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls podName:61c49049-f9ab-48fd-8ee7-7b853de6abd1 nodeName:}" failed. No retries permitted until 2026-09-15 11:25:57.131827196 +0000 UTC m=+58.624604323 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-smdnj" (UID: "61c49049-f9ab-48fd-8ee7-7b853de6abd1") : secret "insights-runtime-extractor-tls" not found
Sep 15 11:25:49.132225 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:49.132209    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61c49049-f9ab-48fd-8ee7-7b853de6abd1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:49.894573 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:49.894514    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:50.046413 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:50.046381    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:50.159812 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:50.159728    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:50.325528 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:50.325496    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:50.479211 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:50.479136    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:50.577080 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:50.577047    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:50.629157 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:50.629128    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:51.001305 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:51.001276    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:51.188429 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:51.188388    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:52.012194 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:52.012152    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:52.200576 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:52.200535    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:52.596949 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:52.596924    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:52.656514 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:52.656488    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:52.797734 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:52.797699    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:53.207198 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:53.207153    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:53.592573 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:53.592537    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:53.798253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:53.798216    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:54.193008 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:54.192975    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:54.246120 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:54.246092    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:54.594594 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:54.594542    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:54.998874 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:54.998785    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:55.059168 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:55.059139    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:55.403329 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:55.403282    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:55.504451 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:55.504412    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:55.607570 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:55.607522    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:55.805515 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:55.805429    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:55.998927 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:55.998885    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:57.028291 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:57.028265    2560 scope.go:117] "RemoveContainer" containerID="417747d0f40cef8f56755b8bc5bea0904cf18f5253c1722270fef35056b036ac"
Sep 15 11:25:57.179006 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:57.178974    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mtm94"
Sep 15 11:25:57.200602 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:57.200576    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:57.202848 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:57.202822    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61c49049-f9ab-48fd-8ee7-7b853de6abd1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-smdnj\" (UID: \"61c49049-f9ab-48fd-8ee7-7b853de6abd1\") " pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:57.301099 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:57.301013    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:25:57.301099 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:57.301087    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-dw8hm" event={"ID":"46058d0a-ac82-4cd3-9482-faf534e13869","Type":"ContainerStarted","Data":"99217f48bafa25464b0de60f0105f4a7f2a5c8026ee18cec765a4a6a12b40a06"}
Sep 15 11:25:57.301409 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:57.301384    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:57.317651 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:57.317606    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-dw8hm" podStartSLOduration=19.503998784 podStartE2EDuration="25.317593512s" totalImagesPullingTime="5.813594728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:25:57.316336917 +0000 UTC m=+58.809114058" watchObservedRunningTime="2026-09-15 11:25:57.317593512 +0000 UTC m=+58.810370660"
Sep 15 11:25:57.486534 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:57.486500    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-smdnj"
Sep 15 11:25:57.500285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:57.500253    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-dw8hm"
Sep 15 11:25:57.763654 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:57.763623    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-smdnj"]
Sep 15 11:25:57.767336 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:25:57.767308    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61c49049_f9ab_48fd_8ee7_7b853de6abd1.slice/crio-71c1a32b291b05ef1c250a0239bb8250516ff207b04b2b5122a7f831a67184b5 WatchSource:0}: Error finding container 71c1a32b291b05ef1c250a0239bb8250516ff207b04b2b5122a7f831a67184b5: Status 404 returned error can't find the container with id 71c1a32b291b05ef1c250a0239bb8250516ff207b04b2b5122a7f831a67184b5
Sep 15 11:25:58.105707 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:58.105679    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:58.208611 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:58.208573    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:58.305717 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:58.305682    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-smdnj" event={"ID":"61c49049-f9ab-48fd-8ee7-7b853de6abd1","Type":"ContainerStarted","Data":"ef71f665f2dd61badad43127995ea6391f511aa8e1549cc93406dc3e67b3d476"}
Sep 15 11:25:58.305872 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:58.305753    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-smdnj" event={"ID":"61c49049-f9ab-48fd-8ee7-7b853de6abd1","Type":"ContainerStarted","Data":"71c1a32b291b05ef1c250a0239bb8250516ff207b04b2b5122a7f831a67184b5"}
Sep 15 11:25:58.552304 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:58.552272    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:58.653140 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:58.653082    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:58.771933 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:58.771886    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:58.970047 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:58.969970    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:59.025576 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:59.025530    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:59.205158 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:59.205121    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:59.311249 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:59.311135    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-smdnj" event={"ID":"61c49049-f9ab-48fd-8ee7-7b853de6abd1","Type":"ContainerStarted","Data":"adeacc21eb45198a1b77968434b1ab75bb361ba56f1b17d4b22d3da6bd6dbf39"}
Sep 15 11:25:59.384153 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:59.384124    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:59.485127 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:59.485095    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:59.603686 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:59.603655    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:25:59.994821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:25:59.994750    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:00.180179 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:00.180146    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:00.316014 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:00.315939    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-smdnj" event={"ID":"61c49049-f9ab-48fd-8ee7-7b853de6abd1","Type":"ContainerStarted","Data":"bfb36982b002aad119d1b17fc8ae2bef3784893af1d21d525173fb4a9a777558"}
Sep 15 11:26:00.332104 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:00.332063    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-smdnj" podStartSLOduration=17.455865351 podStartE2EDuration="19.332047741s" totalImagesPullingTime="1.87618239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 11:25:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:26:00.331669485 +0000 UTC m=+61.824446629" watchObservedRunningTime="2026-09-15 11:26:00.332047741 +0000 UTC m=+61.824824888"
Sep 15 11:26:00.592521 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:00.592488    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:00.801773 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:00.801738    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:00.996626 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:00.996535    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:04.762086 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.762040    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:26:04.762684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.762119    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs\") pod \"network-metrics-daemon-p5wb6\" (UID: \"83354a1c-ecad-424f-8efa-e6bf2e044fae\") " pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:26:04.762684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.762141    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-n7sgg\" (UID: \"e1520a65-2dce-41ce-be10-94731499c0bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"
Sep 15 11:26:04.762684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.762163    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:26:04.764855 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.764831    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1520a65-2dce-41ce-be10-94731499c0bd-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-n7sgg\" (UID: \"e1520a65-2dce-41ce-be10-94731499c0bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"
Sep 15 11:26:04.770255 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.770227    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 11:26:04.770461 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.770441    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls\") pod \"image-registry-784668b6-dtn6x\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") " pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:26:04.770867 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.770852    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 11:26:04.774789 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.774763    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83354a1c-ecad-424f-8efa-e6bf2e044fae-metrics-certs\") pod \"network-metrics-daemon-p5wb6\" (UID: \"83354a1c-ecad-424f-8efa-e6bf2e044fae\") " pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:26:04.774970 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.774953    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b-original-pull-secret\") pod \"global-pull-secret-syncer-4jcwq\" (UID: \"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b\") " pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:26:04.862667 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.862643    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwnxg\" (UniqueName: \"kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg\") pod \"network-check-target-sfjzw\" (UID: \"9674a018-9086-45a1-b5cf-ef8e9dcd0366\") " pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:26:04.862774 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.862677    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:26:04.862774 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.862705    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:26:04.863239 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.863219    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdd8d68-d249-46a5-9eed-37d9cb155861-service-ca-bundle\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:26:04.864711 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.864687    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cdd8d68-d249-46a5-9eed-37d9cb155861-metrics-certs\") pod \"router-default-568d798994-hm7cl\" (UID: \"6cdd8d68-d249-46a5-9eed-37d9cb155861\") " pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:26:04.864949 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.864935    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 11:26:04.875532 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.875515    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 11:26:04.886568 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.886542    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwnxg\" (UniqueName: \"kubernetes.io/projected/9674a018-9086-45a1-b5cf-ef8e9dcd0366-kube-api-access-xwnxg\") pod \"network-check-target-sfjzw\" (UID: \"9674a018-9086-45a1-b5cf-ef8e9dcd0366\") " pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:26:04.950894 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.950864    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4jcwq"
Sep 15 11:26:04.957688 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.957669    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4fvsz\""
Sep 15 11:26:04.962967 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.962949    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n59hg\""
Sep 15 11:26:04.963027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.962962    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7q7fj\" (UID: \"34006888-4d27-47db-a314-26878e17bdff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:26:04.963027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.963001    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:26:04.965025 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.965007    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c09db0b-a1bd-4fe6-9009-567294abff0f-metrics-tls\") pod \"dns-default-c4gb7\" (UID: \"9c09db0b-a1bd-4fe6-9009-567294abff0f\") " pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:26:04.965160 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.965141    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/34006888-4d27-47db-a314-26878e17bdff-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-7q7fj\" (UID: \"34006888-4d27-47db-a314-26878e17bdff\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:26:04.967014 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.966995    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p5wb6"
Sep 15 11:26:04.971996 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.971977    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:26:04.983282 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.983246    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xsd55\""
Sep 15 11:26:04.991096 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:04.991064    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:26:05.020323 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.020068    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wl5v5\""
Sep 15 11:26:05.032016 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.030426    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"
Sep 15 11:26:05.065398 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.065363    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-p5slf\""
Sep 15 11:26:05.074024 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.073997    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:26:05.173236 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.173208    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nnx28\""
Sep 15 11:26:05.182240 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.182220    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"
Sep 15 11:26:05.211537 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.211513    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-55wz6\""
Sep 15 11:26:05.220143 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.220108    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:26:05.224846 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.224824    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4jcwq"]
Sep 15 11:26:05.230318 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:26:05.230289    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f0c4ecd_c730_4a1e_ac07_fdecb4781a4b.slice/crio-af2359f19076a33bbe0238d89b0747d4b098f6498e84f67d6a4507d6d2caf449 WatchSource:0}: Error finding container af2359f19076a33bbe0238d89b0747d4b098f6498e84f67d6a4507d6d2caf449: Status 404 returned error can't find the container with id af2359f19076a33bbe0238d89b0747d4b098f6498e84f67d6a4507d6d2caf449
Sep 15 11:26:05.256190 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.256004    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sfjzw"]
Sep 15 11:26:05.259107 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.259064    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p5wb6"]
Sep 15 11:26:05.259273 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:26:05.259173    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9674a018_9086_45a1_b5cf_ef8e9dcd0366.slice/crio-4cb6e2436c85d23a252df1033d4474b7b160e0dffb3123441f93f7f7b523f345 WatchSource:0}: Error finding container 4cb6e2436c85d23a252df1033d4474b7b160e0dffb3123441f93f7f7b523f345: Status 404 returned error can't find the container with id 4cb6e2436c85d23a252df1033d4474b7b160e0dffb3123441f93f7f7b523f345
Sep 15 11:26:05.262822 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:26:05.262792    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83354a1c_ecad_424f_8efa_e6bf2e044fae.slice/crio-c3d6b01aa7d110bd2bcf4488e7d10dd1ea7ba80a7e77eb96f74aa438e5bf04ba WatchSource:0}: Error finding container c3d6b01aa7d110bd2bcf4488e7d10dd1ea7ba80a7e77eb96f74aa438e5bf04ba: Status 404 returned error can't find the container with id c3d6b01aa7d110bd2bcf4488e7d10dd1ea7ba80a7e77eb96f74aa438e5bf04ba
Sep 15 11:26:05.286659 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.286549    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-784668b6-dtn6x"]
Sep 15 11:26:05.292802 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:26:05.292774    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode93e581e_424f_4960_b2e9_fa94c29f0779.slice/crio-8492c393f6c1deb6ab3649e0a308985059116d8b7fcb8f6daf9be7766ed35c35 WatchSource:0}: Error finding container 8492c393f6c1deb6ab3649e0a308985059116d8b7fcb8f6daf9be7766ed35c35: Status 404 returned error can't find the container with id 8492c393f6c1deb6ab3649e0a308985059116d8b7fcb8f6daf9be7766ed35c35
Sep 15 11:26:05.325163 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.325140    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg"]
Sep 15 11:26:05.332742 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.332714    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-784668b6-dtn6x" event={"ID":"e93e581e-424f-4960-b2e9-fa94c29f0779","Type":"ContainerStarted","Data":"8492c393f6c1deb6ab3649e0a308985059116d8b7fcb8f6daf9be7766ed35c35"}
Sep 15 11:26:05.333863 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.333842    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sfjzw" event={"ID":"9674a018-9086-45a1-b5cf-ef8e9dcd0366","Type":"ContainerStarted","Data":"4cb6e2436c85d23a252df1033d4474b7b160e0dffb3123441f93f7f7b523f345"}
Sep 15 11:26:05.334996 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.334973    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p5wb6" event={"ID":"83354a1c-ecad-424f-8efa-e6bf2e044fae","Type":"ContainerStarted","Data":"c3d6b01aa7d110bd2bcf4488e7d10dd1ea7ba80a7e77eb96f74aa438e5bf04ba"}
Sep 15 11:26:05.336075 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.336058    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4jcwq" event={"ID":"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b","Type":"ContainerStarted","Data":"af2359f19076a33bbe0238d89b0747d4b098f6498e84f67d6a4507d6d2caf449"}
Sep 15 11:26:05.358930 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.358845    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-568d798994-hm7cl"]
Sep 15 11:26:05.361554 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:26:05.361514    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cdd8d68_d249_46a5_9eed_37d9cb155861.slice/crio-96692a977a234328e80bdea732e98cbacdba8dcc5bde48330cad18eb38a15ec1 WatchSource:0}: Error finding container 96692a977a234328e80bdea732e98cbacdba8dcc5bde48330cad18eb38a15ec1: Status 404 returned error can't find the container with id 96692a977a234328e80bdea732e98cbacdba8dcc5bde48330cad18eb38a15ec1
Sep 15 11:26:05.453600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.453548    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj"]
Sep 15 11:26:05.455931 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:26:05.455897    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34006888_4d27_47db_a314_26878e17bdff.slice/crio-d050a0dc8093ff46d810e001bb196520abc55d643b7bdc2cfd5cef86c30b4a33 WatchSource:0}: Error finding container d050a0dc8093ff46d810e001bb196520abc55d643b7bdc2cfd5cef86c30b4a33: Status 404 returned error can't find the container with id d050a0dc8093ff46d810e001bb196520abc55d643b7bdc2cfd5cef86c30b4a33
Sep 15 11:26:05.505140 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:05.505108    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c4gb7"]
Sep 15 11:26:05.509098 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:26:05.509073    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c09db0b_a1bd_4fe6_9009_567294abff0f.slice/crio-1b7026ead15eccff76957b9d0a8c501a3f5992dd79712f0a5ae92d7a2608893b WatchSource:0}: Error finding container 1b7026ead15eccff76957b9d0a8c501a3f5992dd79712f0a5ae92d7a2608893b: Status 404 returned error can't find the container with id 1b7026ead15eccff76957b9d0a8c501a3f5992dd79712f0a5ae92d7a2608893b
Sep 15 11:26:06.347122 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.347022    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg" event={"ID":"e1520a65-2dce-41ce-be10-94731499c0bd","Type":"ContainerStarted","Data":"d6deb74a391a440994f85235a2114eb3642281c798e51be20cd8b4a1f7495a9b"}
Sep 15 11:26:06.350968 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.350872    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-784668b6-dtn6x" event={"ID":"e93e581e-424f-4960-b2e9-fa94c29f0779","Type":"ContainerStarted","Data":"b094744edb7411adfbed9d32830a7b80f6801a3813ffd3893419382e1eabdc60"}
Sep 15 11:26:06.350968 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.350937    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:26:06.353251 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.353172    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c4gb7" event={"ID":"9c09db0b-a1bd-4fe6-9009-567294abff0f","Type":"ContainerStarted","Data":"1b7026ead15eccff76957b9d0a8c501a3f5992dd79712f0a5ae92d7a2608893b"}
Sep 15 11:26:06.355519 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.355451    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj" event={"ID":"34006888-4d27-47db-a314-26878e17bdff","Type":"ContainerStarted","Data":"d050a0dc8093ff46d810e001bb196520abc55d643b7bdc2cfd5cef86c30b4a33"}
Sep 15 11:26:06.358680 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.358168    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-568d798994-hm7cl" event={"ID":"6cdd8d68-d249-46a5-9eed-37d9cb155861","Type":"ContainerStarted","Data":"71f5fed38f045d2c52381af6b84d81f643883956f483bb93d0dae742a1699c2f"}
Sep 15 11:26:06.358680 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.358209    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-568d798994-hm7cl" event={"ID":"6cdd8d68-d249-46a5-9eed-37d9cb155861","Type":"ContainerStarted","Data":"96692a977a234328e80bdea732e98cbacdba8dcc5bde48330cad18eb38a15ec1"}
Sep 15 11:26:06.370936 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.369992    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-784668b6-dtn6x" podStartSLOduration=67.3699734 podStartE2EDuration="1m7.3699734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:24:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:26:06.369029619 +0000 UTC m=+67.861806768" watchObservedRunningTime="2026-09-15 11:26:06.3699734 +0000 UTC m=+67.862750587"
Sep 15 11:26:06.388261 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.387707    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-568d798994-hm7cl" podStartSLOduration=44.387686027 podStartE2EDuration="44.387686027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:25:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:26:06.386692813 +0000 UTC m=+67.879469984" watchObservedRunningTime="2026-09-15 11:26:06.387686027 +0000 UTC m=+67.880463177"
Sep 15 11:26:06.761174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.761028    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:06.773705 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.772442    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-wbz22"]
Sep 15 11:26:06.777844 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.777356    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-wbz22"
Sep 15 11:26:06.784758 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.783583    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-8fwls\""
Sep 15 11:26:06.784758 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.784003    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 15 11:26:06.784758 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.784582    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 15 11:26:06.795208 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.795046    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-wbz22"]
Sep 15 11:26:06.878130 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.878065    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2q2r\" (UniqueName: \"kubernetes.io/projected/e96ecc2f-c511-4b3b-a0cf-888686ce3940-kube-api-access-h2q2r\") pod \"downloads-76f99584f4-wbz22\" (UID: \"e96ecc2f-c511-4b3b-a0cf-888686ce3940\") " pod="openshift-console/downloads-76f99584f4-wbz22"
Sep 15 11:26:06.886197 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.886164    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:06.979181 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.979021    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2q2r\" (UniqueName: \"kubernetes.io/projected/e96ecc2f-c511-4b3b-a0cf-888686ce3940-kube-api-access-h2q2r\") pod \"downloads-76f99584f4-wbz22\" (UID: \"e96ecc2f-c511-4b3b-a0cf-888686ce3940\") " pod="openshift-console/downloads-76f99584f4-wbz22"
Sep 15 11:26:06.981138 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.981093    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:06.990191 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:06.990140    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2q2r\" (UniqueName: \"kubernetes.io/projected/e96ecc2f-c511-4b3b-a0cf-888686ce3940-kube-api-access-h2q2r\") pod \"downloads-76f99584f4-wbz22\" (UID: \"e96ecc2f-c511-4b3b-a0cf-888686ce3940\") " pod="openshift-console/downloads-76f99584f4-wbz22"
Sep 15 11:26:07.074341 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:07.074301    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:26:07.077951 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:07.077652    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:26:07.096988 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:07.096703    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-wbz22"
Sep 15 11:26:07.184827 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:07.182726    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:07.244617 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:07.244448    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:07.368063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:07.367398    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p5wb6" event={"ID":"83354a1c-ecad-424f-8efa-e6bf2e044fae","Type":"ContainerStarted","Data":"4047cd648f6887e3e9f28a6ecb839c74f3a3a37fe2f2b3f4d26f2acfa0788fe2"}
Sep 15 11:26:07.368063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:07.367457    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p5wb6" event={"ID":"83354a1c-ecad-424f-8efa-e6bf2e044fae","Type":"ContainerStarted","Data":"42c4cc245c46dd7ece6b3e938edeb1c3fe83ea82d565a4312fd803758591c6bf"}
Sep 15 11:26:07.368788 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:07.368592    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:26:07.371133 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:07.370657    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-568d798994-hm7cl"
Sep 15 11:26:07.388049 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:07.387533    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-p5wb6" podStartSLOduration=67.083818351 podStartE2EDuration="1m8.387512295s" totalImagesPullingTime="1.303693944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:26:07.386084643 +0000 UTC m=+68.878861792" watchObservedRunningTime="2026-09-15 11:26:07.387512295 +0000 UTC m=+68.880289451"
Sep 15 11:26:07.412841 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:07.412788    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-wbz22"]
Sep 15 11:26:07.727576 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:07.727510    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:07.824548 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:07.824432    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:07.982956 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:26:07.982875    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode96ecc2f_c511_4b3b_a0cf_888686ce3940.slice/crio-c7101c9803c05c4abf09ac339c9468d68716a647d1394491d0c9a4e21eba439f WatchSource:0}: Error finding container c7101c9803c05c4abf09ac339c9468d68716a647d1394491d0c9a4e21eba439f: Status 404 returned error can't find the container with id c7101c9803c05c4abf09ac339c9468d68716a647d1394491d0c9a4e21eba439f
Sep 15 11:26:08.134759 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:08.134718    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:08.190966 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:08.190917    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:08.373034 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:08.372999    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-wbz22" event={"ID":"e96ecc2f-c511-4b3b-a0cf-888686ce3940","Type":"ContainerStarted","Data":"c7101c9803c05c4abf09ac339c9468d68716a647d1394491d0c9a4e21eba439f"}
Sep 15 11:26:08.929681 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:08.929648    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:09.335003 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:09.334922    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:09.435619 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:09.435556    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:10.129311 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:10.129271    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:10.517448 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:10.517415    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:10.618280 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:10.618247    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:11.335640 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:11.335598    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:11.727153 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:11.727110    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:11.831030 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:11.830941    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:11.979027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:11.978943    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:12.062404 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:12.062358    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:12.771756 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:12.771624    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:12.933295 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:12.933251    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:13.066806 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.066710    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:13.387110 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.387061    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:13.399005 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.398977    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4jcwq" event={"ID":"7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b","Type":"ContainerStarted","Data":"cd1957390c76871ca0df904064a5777f99297b7ba6e2640d0f6252fd21ca8ca2"}
Sep 15 11:26:13.401105 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.401073    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg" event={"ID":"e1520a65-2dce-41ce-be10-94731499c0bd","Type":"ContainerStarted","Data":"be67129cbc7800971a01413905bc0ec80f9b1b8d5c22d6b456f20a891e494ec2"}
Sep 15 11:26:13.401223 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.401125    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg" event={"ID":"e1520a65-2dce-41ce-be10-94731499c0bd","Type":"ContainerStarted","Data":"2a980966042f211580dfbd8a530ac031c59dbfa62ed2f50cb9b4164135507e6d"}
Sep 15 11:26:13.402807 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.402781    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sfjzw" event={"ID":"9674a018-9086-45a1-b5cf-ef8e9dcd0366","Type":"ContainerStarted","Data":"f19f69876778623dd2078c7f0e51ad2139f9393c7a9950e77cf014ccbd48c248"}
Sep 15 11:26:13.402918 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.402843    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:26:13.404725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.404702    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c4gb7" event={"ID":"9c09db0b-a1bd-4fe6-9009-567294abff0f","Type":"ContainerStarted","Data":"78d633f897c51880702f14a53f0264c45df19ade8914698a40872662ab0c17b5"}
Sep 15 11:26:13.404847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.404747    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c4gb7" event={"ID":"9c09db0b-a1bd-4fe6-9009-567294abff0f","Type":"ContainerStarted","Data":"6ad6f647ff222797af3be94fb9329616e6a2c224cc643eb7fea1d39165c34781"}
Sep 15 11:26:13.404847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.404803    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:26:13.406313 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.406291    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj" event={"ID":"34006888-4d27-47db-a314-26878e17bdff","Type":"ContainerStarted","Data":"b5a00d21f00b4314410d475b5be80f376035d0c043dd634902ea4d6202a3eea4"}
Sep 15 11:26:13.413697 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.413644    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4jcwq" podStartSLOduration=66.98281601 podStartE2EDuration="1m14.413628223s" totalImagesPullingTime="7.430812213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 11:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:26:13.412846817 +0000 UTC m=+74.905623966" watchObservedRunningTime="2026-09-15 11:26:13.413628223 +0000 UTC m=+74.906405389"
Sep 15 11:26:13.428455 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.428407    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-n7sgg" podStartSLOduration=34.493926007 podStartE2EDuration="41.428394906s" totalImagesPullingTime="6.934468899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:26:13.427619763 +0000 UTC m=+74.920396911" watchObservedRunningTime="2026-09-15 11:26:13.428394906 +0000 UTC m=+74.921172073"
Sep 15 11:26:13.446128 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.446080    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-7q7fj" podStartSLOduration=44.572869858 podStartE2EDuration="51.446065168s" totalImagesPullingTime="6.87319531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:26:13.444650352 +0000 UTC m=+74.937427516" watchObservedRunningTime="2026-09-15 11:26:13.446065168 +0000 UTC m=+74.938842316"
Sep 15 11:26:13.455577 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.455537    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:13.460090 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.460047    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-c4gb7" podStartSLOduration=34.639388762 podStartE2EDuration="41.460030758s" totalImagesPullingTime="6.820641996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:26:13.458796676 +0000 UTC m=+74.951573827" watchObservedRunningTime="2026-09-15 11:26:13.460030758 +0000 UTC m=+74.952807910"
Sep 15 11:26:13.472609 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.472572    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sfjzw" podStartSLOduration=67.392676637 podStartE2EDuration="1m14.47254824s" totalImagesPullingTime="7.079871603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:26:13.471791725 +0000 UTC m=+74.964568874" watchObservedRunningTime="2026-09-15 11:26:13.47254824 +0000 UTC m=+74.965325388"
Sep 15 11:26:13.559431 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.559396    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:13.659611 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:13.659506    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:15.615776 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:15.615733    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:15.694247 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:15.694208    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:15.829751 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:15.829716    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:15.939078 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:15.938984    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:17.968886 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:17.968849    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:18.055541 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:18.055495    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:18.386803 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:18.386759    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:18.487238 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:18.487202    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:21.266807 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:21.266653    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:22.587021 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:22.586980    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:22.699596 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:22.699523    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:22.794388 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:22.794346    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:23.108391 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.108358    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:23.258993 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.258953    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:23.341337 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.341301    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:23.391313 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.391229    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gffmw"]
Sep 15 11:26:23.396295 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.396268    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.398224 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.398203    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 15 11:26:23.398386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.398367    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 15 11:26:23.398472 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.398439    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 15 11:26:23.398922 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.398901    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k5ffd\""
Sep 15 11:26:23.399012 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.398928    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 15 11:26:23.414909 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.414886    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c4gb7"
Sep 15 11:26:23.432901 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.432870    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:23.520674 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.520625    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm4kc\" (UniqueName: \"kubernetes.io/projected/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-kube-api-access-hm4kc\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.520855 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.520693    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-wtmp\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.520855 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.520791    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-root\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.520855 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.520819    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-accelerators-collector-config\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.521020 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.520901    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-sys\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.521020 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.520965    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-textfile\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.521020 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.520993    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-metrics-client-ca\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.521156 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.521028    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-tls\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.521156 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.521088    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.529721 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.529647    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:23.621976 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.621927    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm4kc\" (UniqueName: \"kubernetes.io/projected/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-kube-api-access-hm4kc\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.621976 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.621978    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-wtmp\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.622508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.622004    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-root\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.622508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.622030    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-accelerators-collector-config\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.622508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.622065    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-sys\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.622508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.622099    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-textfile\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.622508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.622126    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-metrics-client-ca\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.622508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.622156    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-tls\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.622508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.622204    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.622508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.622252    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-sys\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.622508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.622279    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-root\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.622508 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:26:23.622312    2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 15 11:26:23.622508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.622345    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-wtmp\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.622508 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:26:23.622367    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-tls podName:776c84f2-fb70-455a-aa6b-ea7cbdbd8889 nodeName:}" failed. No retries permitted until 2026-09-15 11:26:24.122347576 +0000 UTC m=+85.615124702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-tls") pod "node-exporter-gffmw" (UID: "776c84f2-fb70-455a-aa6b-ea7cbdbd8889") : secret "node-exporter-tls" not found
Sep 15 11:26:23.623108 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.622542    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-textfile\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.623108 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.622932    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-metrics-client-ca\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.623108 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.622988    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-accelerators-collector-config\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.624931 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.624912    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.629316 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.629204    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:23.630061 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.630036    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm4kc\" (UniqueName: \"kubernetes.io/projected/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-kube-api-access-hm4kc\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:23.755195 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.755083    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:23.848032 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:23.847988    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:24.024804 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:24.024713    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:24.124908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:24.124870    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:24.128046 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:24.127963    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-tls\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:24.130501 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:24.130477    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/776c84f2-fb70-455a-aa6b-ea7cbdbd8889-node-exporter-tls\") pod \"node-exporter-gffmw\" (UID: \"776c84f2-fb70-455a-aa6b-ea7cbdbd8889\") " pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:24.308315 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:24.308240    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gffmw"
Sep 15 11:26:24.327289 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:24.327262    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:24.421707 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:24.421627    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:24.661403 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:24.661363    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:24.914120 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:24.914030    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:24.979821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:24.979739    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:25.140449 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:25.140414    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:25.178967 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:26:25.178940    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod776c84f2_fb70_455a_aa6b_ea7cbdbd8889.slice/crio-ea5996a47b168b48254f7e69e068729e74324d95dfea8f143f638435e5c416e3 WatchSource:0}: Error finding container ea5996a47b168b48254f7e69e068729e74324d95dfea8f143f638435e5c416e3: Status 404 returned error can't find the container with id ea5996a47b168b48254f7e69e068729e74324d95dfea8f143f638435e5c416e3
Sep 15 11:26:25.235974 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:25.235938    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:25.460224 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:25.460139    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-wbz22" event={"ID":"e96ecc2f-c511-4b3b-a0cf-888686ce3940","Type":"ContainerStarted","Data":"d32acd1f2483b28c41a67b1b5e272c60cabe1475f9783ad458f45cde6bd4fc0e"}
Sep 15 11:26:25.460627 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:25.460600    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-wbz22"
Sep 15 11:26:25.462118 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:25.462088    2560 patch_prober.go:28] interesting pod/downloads-76f99584f4-wbz22 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.18:8080/\": dial tcp 10.132.0.18:8080: connect: connection refused" start-of-body=
Sep 15 11:26:25.462238 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:25.462118    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gffmw" event={"ID":"776c84f2-fb70-455a-aa6b-ea7cbdbd8889","Type":"ContainerStarted","Data":"ea5996a47b168b48254f7e69e068729e74324d95dfea8f143f638435e5c416e3"}
Sep 15 11:26:25.462238 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:25.462156    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-76f99584f4-wbz22" podUID="e96ecc2f-c511-4b3b-a0cf-888686ce3940" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.18:8080/\": dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 15 11:26:25.478597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:25.478536    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-wbz22" podStartSLOduration=2.192012546 podStartE2EDuration="19.478517334s" totalImagesPullingTime="17.286504788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:26:25.477092084 +0000 UTC m=+86.969869235" watchObservedRunningTime="2026-09-15 11:26:25.478517334 +0000 UTC m=+86.971294482"
Sep 15 11:26:25.510421 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:25.510382    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:25.611510 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:25.611470    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:26.300126 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:26.300088    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:26.372036 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:26.372004    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:26.469058 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:26.468974    2560 generic.go:356] "Generic (PLEG): container finished" podID="776c84f2-fb70-455a-aa6b-ea7cbdbd8889" containerID="8e439c56205058aec513a40e54c8ac80bec7a2e271f64f646fb6842a40dcdb49" exitCode=0
Sep 15 11:26:26.469223 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:26.469080    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gffmw" event={"ID":"776c84f2-fb70-455a-aa6b-ea7cbdbd8889","Type":"ContainerDied","Data":"8e439c56205058aec513a40e54c8ac80bec7a2e271f64f646fb6842a40dcdb49"}
Sep 15 11:26:26.486388 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:26.486358    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-wbz22"
Sep 15 11:26:26.650695 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:26.650230    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:26.746226 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:26.746186    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:27.372420 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:27.372390    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:26:27.476621 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:27.476555    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gffmw" event={"ID":"776c84f2-fb70-455a-aa6b-ea7cbdbd8889","Type":"ContainerStarted","Data":"28c366a6770dd8273e51603dbe45b19b4ee0efe09a0daf472b3aaa774d03b1d0"}
Sep 15 11:26:27.476779 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:27.476654    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gffmw" event={"ID":"776c84f2-fb70-455a-aa6b-ea7cbdbd8889","Type":"ContainerStarted","Data":"531f7eb892a46dc88ea6e1d63aab73ef954170aae5d5e24ea0607c0680fe6b87"}
Sep 15 11:26:27.495433 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:27.495382    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gffmw" podStartSLOduration=3.42277094 podStartE2EDuration="4.495364503s" totalImagesPullingTime="991.970711ms" totalInitContainerRuntime="80.622852ms" isStatefulPod=true podCreationTimestamp="2026-09-15 11:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:26:27.49533459 +0000 UTC m=+88.988111756" watchObservedRunningTime="2026-09-15 11:26:27.495364503 +0000 UTC m=+88.988141653"
Sep 15 11:26:27.545059 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:27.545011    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:27.648298 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:27.648202    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:28.183968 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:28.183930    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:28.263408 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:28.263372    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:28.563166 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:28.563063    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:28.658891 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:28.658853    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:28.936391 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:28.936354    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:29.367388 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.367351    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-784668b6-dtn6x"]
Sep 15 11:26:29.502918 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.502863    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:29.601292 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.601250    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:29.666139 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.666059    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 11:26:29.689282 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.689204    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:29.690070 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.690044    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 11:26:29.690187 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.690139    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.693657 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.693483    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 15 11:26:29.693657 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.693507    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 15 11:26:29.693894 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.693873    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 15 11:26:29.694278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.693926    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 15 11:26:29.694449 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.693982    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 15 11:26:29.694532 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.693983    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 15 11:26:29.694616 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.694542    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 15 11:26:29.694616 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.694037    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 15 11:26:29.694718 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.694098    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 15 11:26:29.694718 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.694120    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 15 11:26:29.694816 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.694122    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 15 11:26:29.694816 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.694784    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 15 11:26:29.694816 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.694178    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d208n1b14g9ge\""
Sep 15 11:26:29.694960 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.694155    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 15 11:26:29.694960 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.694915    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-cp4vc\""
Sep 15 11:26:29.696885 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.696709    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 15 11:26:29.701305 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.701286    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 15 11:26:29.878407 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.878366    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/126d14d7-717b-4929-8774-b0575253ea5f-config-out\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.878609 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.878415    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.878609 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.878484    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-web-config\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.878609 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.878520    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.878609 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.878581    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.878821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.878608    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.878821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.878652    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.878821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.878670    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.878821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.878719    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.878821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.878747    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.878821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.878792    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.879084 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.878827    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-config\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.879084 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.878899    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.879084 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.878947    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.879084 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.878982    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/126d14d7-717b-4929-8774-b0575253ea5f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.879084 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.879012    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.879084 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.879054    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.879336 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.879089    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.879336 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.879124    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.879336 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.879140    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8ncq\" (UniqueName: \"kubernetes.io/projected/126d14d7-717b-4929-8774-b0575253ea5f-kube-api-access-x8ncq\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980011 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.979938    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-config\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980011 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.979981    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980019    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980048    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/126d14d7-717b-4929-8774-b0575253ea5f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980074    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980097    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980123    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980148    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980168    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8ncq\" (UniqueName: \"kubernetes.io/projected/126d14d7-717b-4929-8774-b0575253ea5f-kube-api-access-x8ncq\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980197    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/126d14d7-717b-4929-8774-b0575253ea5f-config-out\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980219    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980248    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-web-config\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980272    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980296    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980318    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980359    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980384    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980426    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980465    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.980710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980491    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.981108 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980804    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.981108 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.980821    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.982228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.982201    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.983276 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.983254    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.983712 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.983692    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.984006 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.983980    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/126d14d7-717b-4929-8774-b0575253ea5f-config-out\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.984188 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.984160    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.984603 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.984576    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-config\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.984699 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.984645    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.984762 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.984741    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.985435 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.985410    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.985620 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.985597    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.985987 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.985962    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.986975 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.986950    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.988909 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.988814    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-web-config\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.989050 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.988947    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/126d14d7-717b-4929-8774-b0575253ea5f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.989126 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.988946    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.989181 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.989155    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.990534 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.990509    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8ncq\" (UniqueName: \"kubernetes.io/projected/126d14d7-717b-4929-8774-b0575253ea5f-kube-api-access-x8ncq\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:29.990737 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:29.990510    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:30.003377 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:30.003354    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:30.317194 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:30.317165    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 11:26:30.319264 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:26:30.319221    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod126d14d7_717b_4929_8774_b0575253ea5f.slice/crio-74b09e0ad9224b08b0e947f15475ef84f726116a7f78cb4adfc900e8f407ece8 WatchSource:0}: Error finding container 74b09e0ad9224b08b0e947f15475ef84f726116a7f78cb4adfc900e8f407ece8: Status 404 returned error can't find the container with id 74b09e0ad9224b08b0e947f15475ef84f726116a7f78cb4adfc900e8f407ece8
Sep 15 11:26:30.424966 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:30.424923    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:30.496790 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:30.496750    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"126d14d7-717b-4929-8774-b0575253ea5f","Type":"ContainerStarted","Data":"74b09e0ad9224b08b0e947f15475ef84f726116a7f78cb4adfc900e8f407ece8"}
Sep 15 11:26:30.523598 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:30.523553    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:32.508498 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:32.508468    2560 generic.go:356] "Generic (PLEG): container finished" podID="126d14d7-717b-4929-8774-b0575253ea5f" containerID="da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09" exitCode=0
Sep 15 11:26:32.509027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:32.508524    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"126d14d7-717b-4929-8774-b0575253ea5f","Type":"ContainerDied","Data":"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09"}
Sep 15 11:26:32.604332 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:32.604296    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:32.694185 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:32.694137    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:36.530354 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:36.530282    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"126d14d7-717b-4929-8774-b0575253ea5f","Type":"ContainerStarted","Data":"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"}
Sep 15 11:26:36.530724 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:36.530370    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"126d14d7-717b-4929-8774-b0575253ea5f","Type":"ContainerStarted","Data":"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"}
Sep 15 11:26:36.876951 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:36.876915    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:36.979289 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:36.979235    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:37.081647 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:37.081613    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:37.178535 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:37.178467    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:37.279075 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:37.279042    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:39.054480 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:39.054439    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:39.125446 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:39.125384    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:40.548486 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:40.548411    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"126d14d7-717b-4929-8774-b0575253ea5f","Type":"ContainerStarted","Data":"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"}
Sep 15 11:26:40.548878 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:40.548547    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"126d14d7-717b-4929-8774-b0575253ea5f","Type":"ContainerStarted","Data":"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"}
Sep 15 11:26:40.548878 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:40.548621    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"126d14d7-717b-4929-8774-b0575253ea5f","Type":"ContainerStarted","Data":"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"}
Sep 15 11:26:40.548878 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:40.548662    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"126d14d7-717b-4929-8774-b0575253ea5f","Type":"ContainerStarted","Data":"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"}
Sep 15 11:26:40.580682 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:40.580431    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.899162551 podStartE2EDuration="11.580409635s" totalImagesPullingTime="8.657071299s" totalInitContainerRuntime="24.175785ms" isStatefulPod=false podCreationTimestamp="2026-09-15 11:26:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:26:40.578350996 +0000 UTC m=+102.071128148" watchObservedRunningTime="2026-09-15 11:26:40.580409635 +0000 UTC m=+102.073186782"
Sep 15 11:26:41.576974 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:41.576935    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:41.635575 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:41.635531    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:43.551920 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:43.551885    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:43.652005 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:43.651960    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:43.752088 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:43.752054    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:44.414368 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:44.414341    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sfjzw"
Sep 15 11:26:45.004218 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:45.004184    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:26:45.128450 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:45.128418    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:54.390479 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.390438    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-784668b6-dtn6x" podUID="e93e581e-424f-4960-b2e9-fa94c29f0779" containerName="registry" containerID="cri-o://b094744edb7411adfbed9d32830a7b80f6801a3813ffd3893419382e1eabdc60" gracePeriod=30
Sep 15 11:26:54.601882 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.601849    2560 generic.go:356] "Generic (PLEG): container finished" podID="e93e581e-424f-4960-b2e9-fa94c29f0779" containerID="b094744edb7411adfbed9d32830a7b80f6801a3813ffd3893419382e1eabdc60" exitCode=0
Sep 15 11:26:54.602022 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.601923    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-784668b6-dtn6x" event={"ID":"e93e581e-424f-4960-b2e9-fa94c29f0779","Type":"ContainerDied","Data":"b094744edb7411adfbed9d32830a7b80f6801a3813ffd3893419382e1eabdc60"}
Sep 15 11:26:54.636228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.636198    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:26:54.671856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.671780    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-certificates\") pod \"e93e581e-424f-4960-b2e9-fa94c29f0779\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") "
Sep 15 11:26:54.671856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.671821    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e93e581e-424f-4960-b2e9-fa94c29f0779-image-registry-private-configuration\") pod \"e93e581e-424f-4960-b2e9-fa94c29f0779\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") "
Sep 15 11:26:54.671856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.671851    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls\") pod \"e93e581e-424f-4960-b2e9-fa94c29f0779\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") "
Sep 15 11:26:54.672112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.671866    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e93e581e-424f-4960-b2e9-fa94c29f0779-trusted-ca\") pod \"e93e581e-424f-4960-b2e9-fa94c29f0779\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") "
Sep 15 11:26:54.672112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.671894    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e93e581e-424f-4960-b2e9-fa94c29f0779-ca-trust-extracted\") pod \"e93e581e-424f-4960-b2e9-fa94c29f0779\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") "
Sep 15 11:26:54.672112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.671928    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-bound-sa-token\") pod \"e93e581e-424f-4960-b2e9-fa94c29f0779\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") "
Sep 15 11:26:54.672112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.671946    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66vdv\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-kube-api-access-66vdv\") pod \"e93e581e-424f-4960-b2e9-fa94c29f0779\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") "
Sep 15 11:26:54.672112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.671970    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e93e581e-424f-4960-b2e9-fa94c29f0779-installation-pull-secrets\") pod \"e93e581e-424f-4960-b2e9-fa94c29f0779\" (UID: \"e93e581e-424f-4960-b2e9-fa94c29f0779\") "
Sep 15 11:26:54.672371 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.672199    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e93e581e-424f-4960-b2e9-fa94c29f0779" (UID: "e93e581e-424f-4960-b2e9-fa94c29f0779"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:26:54.672500 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.672475    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e93e581e-424f-4960-b2e9-fa94c29f0779-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e93e581e-424f-4960-b2e9-fa94c29f0779" (UID: "e93e581e-424f-4960-b2e9-fa94c29f0779"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:26:54.674426 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.674400    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e93e581e-424f-4960-b2e9-fa94c29f0779" (UID: "e93e581e-424f-4960-b2e9-fa94c29f0779"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:26:54.674687 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.674659    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e93e581e-424f-4960-b2e9-fa94c29f0779" (UID: "e93e581e-424f-4960-b2e9-fa94c29f0779"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:26:54.674784 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.674679    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e93e581e-424f-4960-b2e9-fa94c29f0779-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e93e581e-424f-4960-b2e9-fa94c29f0779" (UID: "e93e581e-424f-4960-b2e9-fa94c29f0779"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:26:54.674784 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.674768    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-kube-api-access-66vdv" (OuterVolumeSpecName: "kube-api-access-66vdv") pod "e93e581e-424f-4960-b2e9-fa94c29f0779" (UID: "e93e581e-424f-4960-b2e9-fa94c29f0779"). InnerVolumeSpecName "kube-api-access-66vdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:26:54.674895 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.674797    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e93e581e-424f-4960-b2e9-fa94c29f0779-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e93e581e-424f-4960-b2e9-fa94c29f0779" (UID: "e93e581e-424f-4960-b2e9-fa94c29f0779"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:26:54.681458 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.681429    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e93e581e-424f-4960-b2e9-fa94c29f0779-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e93e581e-424f-4960-b2e9-fa94c29f0779" (UID: "e93e581e-424f-4960-b2e9-fa94c29f0779"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:26:54.773214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.773174    2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-bound-sa-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:26:54.773214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.773210    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66vdv\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-kube-api-access-66vdv\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:26:54.773214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.773220    2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e93e581e-424f-4960-b2e9-fa94c29f0779-installation-pull-secrets\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:26:54.773214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.773230    2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-certificates\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:26:54.773439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.773240    2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e93e581e-424f-4960-b2e9-fa94c29f0779-image-registry-private-configuration\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:26:54.773439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.773250    2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e93e581e-424f-4960-b2e9-fa94c29f0779-registry-tls\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:26:54.773439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.773258    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e93e581e-424f-4960-b2e9-fa94c29f0779-trusted-ca\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:26:54.773439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:54.773265    2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e93e581e-424f-4960-b2e9-fa94c29f0779-ca-trust-extracted\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:26:55.606708 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:55.606671    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-784668b6-dtn6x" event={"ID":"e93e581e-424f-4960-b2e9-fa94c29f0779","Type":"ContainerDied","Data":"8492c393f6c1deb6ab3649e0a308985059116d8b7fcb8f6daf9be7766ed35c35"}
Sep 15 11:26:55.606708 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:55.606694    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-784668b6-dtn6x"
Sep 15 11:26:55.607164 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:55.606730    2560 scope.go:117] "RemoveContainer" containerID="b094744edb7411adfbed9d32830a7b80f6801a3813ffd3893419382e1eabdc60"
Sep 15 11:26:55.621978 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:55.621952    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-784668b6-dtn6x"]
Sep 15 11:26:55.623964 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:55.623941    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-784668b6-dtn6x"]
Sep 15 11:26:56.612853 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:56.612821    2560 generic.go:356] "Generic (PLEG): container finished" podID="2667a7e4-1546-43a1-b61d-01f8eefe3c82" containerID="220ce69b91a0e4ecb7e4ea54eabf272d95893f742e0dd3c4a6c55ffe768ef266" exitCode=0
Sep 15 11:26:56.613218 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:56.612861    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-njsmc" event={"ID":"2667a7e4-1546-43a1-b61d-01f8eefe3c82","Type":"ContainerDied","Data":"220ce69b91a0e4ecb7e4ea54eabf272d95893f742e0dd3c4a6c55ffe768ef266"}
Sep 15 11:26:56.613218 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:56.613185    2560 scope.go:117] "RemoveContainer" containerID="220ce69b91a0e4ecb7e4ea54eabf272d95893f742e0dd3c4a6c55ffe768ef266"
Sep 15 11:26:57.033758 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:57.033676    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e93e581e-424f-4960-b2e9-fa94c29f0779" path="/var/lib/kubelet/pods/e93e581e-424f-4960-b2e9-fa94c29f0779/volumes"
Sep 15 11:26:57.618099 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:57.618069    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-njsmc" event={"ID":"2667a7e4-1546-43a1-b61d-01f8eefe3c82","Type":"ContainerStarted","Data":"b45ddc6d32b20b65d84677e17d43569437ab75dca67f04efd4940e1bc9f4f609"}
Sep 15 11:26:59.572355 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:59.572321    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:26:59.628908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:26:59.628873    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:27:02.231699 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:02.231666    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:27:06.652663 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:06.652628    2560 generic.go:356] "Generic (PLEG): container finished" podID="c8329b83-30a0-4ead-9c95-57f0916610aa" containerID="96d3ca608813e6688541414c57f6103b272f866d70c115c8e93442646fe9634c" exitCode=0
Sep 15 11:27:06.653080 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:06.652705    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5" event={"ID":"c8329b83-30a0-4ead-9c95-57f0916610aa","Type":"ContainerDied","Data":"96d3ca608813e6688541414c57f6103b272f866d70c115c8e93442646fe9634c"}
Sep 15 11:27:06.653137 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:06.653109    2560 scope.go:117] "RemoveContainer" containerID="96d3ca608813e6688541414c57f6103b272f866d70c115c8e93442646fe9634c"
Sep 15 11:27:06.654322 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:06.654300    2560 generic.go:356] "Generic (PLEG): container finished" podID="21196aa4-a811-43f9-b006-1f448fce3ea7" containerID="342d9c60210683a45d98167b7e41676d53677fe27643060ab6e30500722ec727" exitCode=0
Sep 15 11:27:06.654415 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:06.654361    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z" event={"ID":"21196aa4-a811-43f9-b006-1f448fce3ea7","Type":"ContainerDied","Data":"342d9c60210683a45d98167b7e41676d53677fe27643060ab6e30500722ec727"}
Sep 15 11:27:06.654618 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:06.654606    2560 scope.go:117] "RemoveContainer" containerID="342d9c60210683a45d98167b7e41676d53677fe27643060ab6e30500722ec727"
Sep 15 11:27:07.659964 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:07.659930    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-7zt9z" event={"ID":"21196aa4-a811-43f9-b006-1f448fce3ea7","Type":"ContainerStarted","Data":"f71d77b91c1f123839d3339305bc3906bde70bf7c6f28f0b0399924a446f0ad7"}
Sep 15 11:27:07.661854 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:07.661817    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-n5zv5" event={"ID":"c8329b83-30a0-4ead-9c95-57f0916610aa","Type":"ContainerStarted","Data":"c96a745b4bff95f47492a444afd2843781a58ce355bc56c86a388e80a9b4f102"}
Sep 15 11:27:29.458961 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:29.458926    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:27:29.527412 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:29.527387    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:27:30.003726 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:30.003695    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:30.019168 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:30.019139    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:30.764175 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:30.764145    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:39.465856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:39.465813    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:27:39.520311 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:39.520272    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:27:47.951828 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:47.951793    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 11:27:47.952222 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:47.952113    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="kube-rbac-proxy" containerID="cri-o://abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117" gracePeriod=600
Sep 15 11:27:47.952222 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:47.952156    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="thanos-sidecar" containerID="cri-o://287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990" gracePeriod=600
Sep 15 11:27:47.952222 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:47.952107    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="prometheus" containerID="cri-o://508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8" gracePeriod=600
Sep 15 11:27:47.952380 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:47.952177    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="kube-rbac-proxy-thanos" containerID="cri-o://a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4" gracePeriod=600
Sep 15 11:27:47.952380 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:47.952173    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="config-reloader" containerID="cri-o://d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d" gracePeriod=600
Sep 15 11:27:47.952380 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:47.952173    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="kube-rbac-proxy-web" containerID="cri-o://eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4" gracePeriod=600
Sep 15 11:27:48.201086 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.201061    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.294009 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.293924    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/126d14d7-717b-4929-8774-b0575253ea5f-config-out\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294009 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.293973    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294009 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294004    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8ncq\" (UniqueName: \"kubernetes.io/projected/126d14d7-717b-4929-8774-b0575253ea5f-kube-api-access-x8ncq\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294287 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294049    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-serving-certs-ca-bundle\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294287 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294086    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-config\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294287 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294117    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-thanos-prometheus-http-client-file\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294287 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294150    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-metrics-client-certs\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294287 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294173    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-metrics-client-ca\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294287 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294202    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-1\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294287 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294229    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-2\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294287 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294255    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-web-config\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294287 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294281    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294731 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294311    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-tls\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294731 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294338    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/126d14d7-717b-4929-8774-b0575253ea5f-tls-assets\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294731 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294376    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-kubelet-serving-ca-bundle\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294731 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294411    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-db\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294731 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294435    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-grpc-tls\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294731 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294460    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-kube-rbac-proxy\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294731 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294485    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-trusted-ca-bundle\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.294731 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294486    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:27:48.294731 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294513    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-0\") pod \"126d14d7-717b-4929-8774-b0575253ea5f\" (UID: \"126d14d7-717b-4929-8774-b0575253ea5f\") "
Sep 15 11:27:48.295154 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.294784    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.295154 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.295103    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:27:48.295265 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.295196    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:27:48.295265 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.295238    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:27:48.295760 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.295411    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:27:48.296993 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.296968    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-config" (OuterVolumeSpecName: "config") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:27:48.297129 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.297048    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126d14d7-717b-4929-8774-b0575253ea5f-config-out" (OuterVolumeSpecName: "config-out") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:27:48.298033 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.297838    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:27:48.298033 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.297927    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:27:48.298033 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.297976    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:27:48.298033 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.297998    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/126d14d7-717b-4929-8774-b0575253ea5f-kube-api-access-x8ncq" (OuterVolumeSpecName: "kube-api-access-x8ncq") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "kube-api-access-x8ncq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:27:48.298502 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.298475    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:27:48.298594 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.298502    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:27:48.298805 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.298774    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:27:48.299103 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.299074    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:27:48.299461 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.299442    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:27:48.299520 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.299480    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/126d14d7-717b-4929-8774-b0575253ea5f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:27:48.299520 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.299510    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:27:48.299682 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.299666    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:27:48.308472 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.308450    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-web-config" (OuterVolumeSpecName: "web-config") pod "126d14d7-717b-4929-8774-b0575253ea5f" (UID: "126d14d7-717b-4929-8774-b0575253ea5f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:27:48.396163 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396123    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396163 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396151    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-db\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396163 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396162    2560 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-grpc-tls\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396163 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396171    2560 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-kube-rbac-proxy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396181    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396190    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396198    2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/126d14d7-717b-4929-8774-b0575253ea5f-config-out\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396207    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396215    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8ncq\" (UniqueName: \"kubernetes.io/projected/126d14d7-717b-4929-8774-b0575253ea5f-kube-api-access-x8ncq\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396224    2560 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-config\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396232    2560 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396241    2560 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-metrics-client-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396249    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-configmap-metrics-client-ca\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396257    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396265    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/126d14d7-717b-4929-8774-b0575253ea5f-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396273    2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-web-config\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396281    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396289    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/126d14d7-717b-4929-8774-b0575253ea5f-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.396432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.396297    2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/126d14d7-717b-4929-8774-b0575253ea5f-tls-assets\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:27:48.818897 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.818862    2560 generic.go:356] "Generic (PLEG): container finished" podID="126d14d7-717b-4929-8774-b0575253ea5f" containerID="a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4" exitCode=0
Sep 15 11:27:48.818897 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.818894    2560 generic.go:356] "Generic (PLEG): container finished" podID="126d14d7-717b-4929-8774-b0575253ea5f" containerID="abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117" exitCode=0
Sep 15 11:27:48.818897 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.818903    2560 generic.go:356] "Generic (PLEG): container finished" podID="126d14d7-717b-4929-8774-b0575253ea5f" containerID="eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4" exitCode=0
Sep 15 11:27:48.819115 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.818912    2560 generic.go:356] "Generic (PLEG): container finished" podID="126d14d7-717b-4929-8774-b0575253ea5f" containerID="287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990" exitCode=0
Sep 15 11:27:48.819115 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.818920    2560 generic.go:356] "Generic (PLEG): container finished" podID="126d14d7-717b-4929-8774-b0575253ea5f" containerID="d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d" exitCode=0
Sep 15 11:27:48.819115 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.818928    2560 generic.go:356] "Generic (PLEG): container finished" podID="126d14d7-717b-4929-8774-b0575253ea5f" containerID="508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8" exitCode=0
Sep 15 11:27:48.819115 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.818956    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.819115 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.818951    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"126d14d7-717b-4929-8774-b0575253ea5f","Type":"ContainerDied","Data":"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"}
Sep 15 11:27:48.819115 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.819102    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"126d14d7-717b-4929-8774-b0575253ea5f","Type":"ContainerDied","Data":"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"}
Sep 15 11:27:48.819305 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.819147    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"126d14d7-717b-4929-8774-b0575253ea5f","Type":"ContainerDied","Data":"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"}
Sep 15 11:27:48.819305 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.819167    2560 scope.go:117] "RemoveContainer" containerID="a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"
Sep 15 11:27:48.819305 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.819194    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"126d14d7-717b-4929-8774-b0575253ea5f","Type":"ContainerDied","Data":"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"}
Sep 15 11:27:48.819305 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.819236    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"126d14d7-717b-4929-8774-b0575253ea5f","Type":"ContainerDied","Data":"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"}
Sep 15 11:27:48.819305 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.819277    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"126d14d7-717b-4929-8774-b0575253ea5f","Type":"ContainerDied","Data":"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"}
Sep 15 11:27:48.819529 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.819314    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"126d14d7-717b-4929-8774-b0575253ea5f","Type":"ContainerDied","Data":"74b09e0ad9224b08b0e947f15475ef84f726116a7f78cb4adfc900e8f407ece8"}
Sep 15 11:27:48.827910 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.827891    2560 scope.go:117] "RemoveContainer" containerID="abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"
Sep 15 11:27:48.839046 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.839025    2560 scope.go:117] "RemoveContainer" containerID="eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"
Sep 15 11:27:48.845263 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.845239    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 11:27:48.846091 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.846074    2560 scope.go:117] "RemoveContainer" containerID="287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"
Sep 15 11:27:48.850934 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.850912    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 11:27:48.853392 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.853374    2560 scope.go:117] "RemoveContainer" containerID="d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"
Sep 15 11:27:48.860301 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.860285    2560 scope.go:117] "RemoveContainer" containerID="508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"
Sep 15 11:27:48.866981 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.866965    2560 scope.go:117] "RemoveContainer" containerID="da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09"
Sep 15 11:27:48.873322 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.873295    2560 scope.go:117] "RemoveContainer" containerID="a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"
Sep 15 11:27:48.873616 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:27:48.873581    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4\": container with ID starting with a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4 not found: ID does not exist" containerID="a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"
Sep 15 11:27:48.873666 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.873625    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"} err="failed to get container status \"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4\": rpc error: code = NotFound desc = could not find container \"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4\": container with ID starting with a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4 not found: ID does not exist"
Sep 15 11:27:48.873666 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.873656    2560 scope.go:117] "RemoveContainer" containerID="abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"
Sep 15 11:27:48.873916 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:27:48.873898    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117\": container with ID starting with abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117 not found: ID does not exist" containerID="abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"
Sep 15 11:27:48.873983 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.873926    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"} err="failed to get container status \"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117\": rpc error: code = NotFound desc = could not find container \"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117\": container with ID starting with abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117 not found: ID does not exist"
Sep 15 11:27:48.873983 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.873948    2560 scope.go:117] "RemoveContainer" containerID="eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"
Sep 15 11:27:48.874206 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:27:48.874189    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4\": container with ID starting with eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4 not found: ID does not exist" containerID="eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"
Sep 15 11:27:48.874244 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.874211    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"} err="failed to get container status \"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4\": rpc error: code = NotFound desc = could not find container \"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4\": container with ID starting with eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4 not found: ID does not exist"
Sep 15 11:27:48.874244 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.874226    2560 scope.go:117] "RemoveContainer" containerID="287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"
Sep 15 11:27:48.874429 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:27:48.874414    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990\": container with ID starting with 287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990 not found: ID does not exist" containerID="287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"
Sep 15 11:27:48.874470 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.874432    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"} err="failed to get container status \"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990\": rpc error: code = NotFound desc = could not find container \"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990\": container with ID starting with 287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990 not found: ID does not exist"
Sep 15 11:27:48.874470 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.874443    2560 scope.go:117] "RemoveContainer" containerID="d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"
Sep 15 11:27:48.874692 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:27:48.874675    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d\": container with ID starting with d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d not found: ID does not exist" containerID="d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"
Sep 15 11:27:48.874762 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.874697    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"} err="failed to get container status \"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d\": rpc error: code = NotFound desc = could not find container \"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d\": container with ID starting with d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d not found: ID does not exist"
Sep 15 11:27:48.874762 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.874716    2560 scope.go:117] "RemoveContainer" containerID="508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"
Sep 15 11:27:48.874961 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:27:48.874944    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8\": container with ID starting with 508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8 not found: ID does not exist" containerID="508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"
Sep 15 11:27:48.875004 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.874965    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"} err="failed to get container status \"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8\": rpc error: code = NotFound desc = could not find container \"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8\": container with ID starting with 508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8 not found: ID does not exist"
Sep 15 11:27:48.875004 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.874978    2560 scope.go:117] "RemoveContainer" containerID="da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09"
Sep 15 11:27:48.875211 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:27:48.875194    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09\": container with ID starting with da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09 not found: ID does not exist" containerID="da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09"
Sep 15 11:27:48.875253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.875215    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09"} err="failed to get container status \"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09\": rpc error: code = NotFound desc = could not find container \"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09\": container with ID starting with da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09 not found: ID does not exist"
Sep 15 11:27:48.875253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.875228    2560 scope.go:117] "RemoveContainer" containerID="a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"
Sep 15 11:27:48.875436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.875419    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"} err="failed to get container status \"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4\": rpc error: code = NotFound desc = could not find container \"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4\": container with ID starting with a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4 not found: ID does not exist"
Sep 15 11:27:48.875501 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.875438    2560 scope.go:117] "RemoveContainer" containerID="abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"
Sep 15 11:27:48.875764 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.875743    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"} err="failed to get container status \"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117\": rpc error: code = NotFound desc = could not find container \"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117\": container with ID starting with abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117 not found: ID does not exist"
Sep 15 11:27:48.875828 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.875771    2560 scope.go:117] "RemoveContainer" containerID="eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"
Sep 15 11:27:48.876008 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.875988    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"} err="failed to get container status \"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4\": rpc error: code = NotFound desc = could not find container \"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4\": container with ID starting with eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4 not found: ID does not exist"
Sep 15 11:27:48.876086 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.876009    2560 scope.go:117] "RemoveContainer" containerID="287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"
Sep 15 11:27:48.876238 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.876219    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"} err="failed to get container status \"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990\": rpc error: code = NotFound desc = could not find container \"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990\": container with ID starting with 287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990 not found: ID does not exist"
Sep 15 11:27:48.876277 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.876238    2560 scope.go:117] "RemoveContainer" containerID="d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"
Sep 15 11:27:48.876452 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.876434    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"} err="failed to get container status \"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d\": rpc error: code = NotFound desc = could not find container \"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d\": container with ID starting with d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d not found: ID does not exist"
Sep 15 11:27:48.876518 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.876453    2560 scope.go:117] "RemoveContainer" containerID="508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"
Sep 15 11:27:48.876722 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.876704    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"} err="failed to get container status \"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8\": rpc error: code = NotFound desc = could not find container \"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8\": container with ID starting with 508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8 not found: ID does not exist"
Sep 15 11:27:48.876806 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.876722    2560 scope.go:117] "RemoveContainer" containerID="da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09"
Sep 15 11:27:48.876926 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.876912    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09"} err="failed to get container status \"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09\": rpc error: code = NotFound desc = could not find container \"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09\": container with ID starting with da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09 not found: ID does not exist"
Sep 15 11:27:48.876966 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.876925    2560 scope.go:117] "RemoveContainer" containerID="a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"
Sep 15 11:27:48.877122 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.877086    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"} err="failed to get container status \"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4\": rpc error: code = NotFound desc = could not find container \"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4\": container with ID starting with a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4 not found: ID does not exist"
Sep 15 11:27:48.877122 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.877113    2560 scope.go:117] "RemoveContainer" containerID="abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"
Sep 15 11:27:48.877510 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.877450    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"} err="failed to get container status \"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117\": rpc error: code = NotFound desc = could not find container \"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117\": container with ID starting with abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117 not found: ID does not exist"
Sep 15 11:27:48.877510 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.877476    2560 scope.go:117] "RemoveContainer" containerID="eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"
Sep 15 11:27:48.878032 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.878000    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"} err="failed to get container status \"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4\": rpc error: code = NotFound desc = could not find container \"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4\": container with ID starting with eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4 not found: ID does not exist"
Sep 15 11:27:48.878032 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.878031    2560 scope.go:117] "RemoveContainer" containerID="287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"
Sep 15 11:27:48.878341 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.878323    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"} err="failed to get container status \"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990\": rpc error: code = NotFound desc = could not find container \"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990\": container with ID starting with 287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990 not found: ID does not exist"
Sep 15 11:27:48.878381 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.878343    2560 scope.go:117] "RemoveContainer" containerID="d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"
Sep 15 11:27:48.878620 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.878597    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"} err="failed to get container status \"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d\": rpc error: code = NotFound desc = could not find container \"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d\": container with ID starting with d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d not found: ID does not exist"
Sep 15 11:27:48.878674 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.878621    2560 scope.go:117] "RemoveContainer" containerID="508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"
Sep 15 11:27:48.878892 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.878877    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"} err="failed to get container status \"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8\": rpc error: code = NotFound desc = could not find container \"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8\": container with ID starting with 508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8 not found: ID does not exist"
Sep 15 11:27:48.878931 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.878892    2560 scope.go:117] "RemoveContainer" containerID="da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09"
Sep 15 11:27:48.879130 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879111    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09"} err="failed to get container status \"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09\": rpc error: code = NotFound desc = could not find container \"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09\": container with ID starting with da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09 not found: ID does not exist"
Sep 15 11:27:48.879181 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879131    2560 scope.go:117] "RemoveContainer" containerID="a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"
Sep 15 11:27:48.879247 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879229    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 11:27:48.879345 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879327    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"} err="failed to get container status \"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4\": rpc error: code = NotFound desc = could not find container \"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4\": container with ID starting with a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4 not found: ID does not exist"
Sep 15 11:27:48.879388 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879346    2560 scope.go:117] "RemoveContainer" containerID="abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"
Sep 15 11:27:48.879533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879517    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="thanos-sidecar"
Sep 15 11:27:48.879590 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879535    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="thanos-sidecar"
Sep 15 11:27:48.879590 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879534    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"} err="failed to get container status \"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117\": rpc error: code = NotFound desc = could not find container \"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117\": container with ID starting with abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117 not found: ID does not exist"
Sep 15 11:27:48.879590 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879545    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="kube-rbac-proxy-thanos"
Sep 15 11:27:48.879590 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879552    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="kube-rbac-proxy-thanos"
Sep 15 11:27:48.879590 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879552    2560 scope.go:117] "RemoveContainer" containerID="eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"
Sep 15 11:27:48.879590 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879581    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="prometheus"
Sep 15 11:27:48.879590 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879588    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="prometheus"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879598    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="init-config-reloader"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879604    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="init-config-reloader"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879611    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="config-reloader"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879619    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="config-reloader"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879633    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="kube-rbac-proxy"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879640    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="kube-rbac-proxy"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879651    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e93e581e-424f-4960-b2e9-fa94c29f0779" containerName="registry"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879659    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e93e581e-424f-4960-b2e9-fa94c29f0779" containerName="registry"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879668    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="kube-rbac-proxy-web"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879675    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="kube-rbac-proxy-web"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879748    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="kube-rbac-proxy-thanos"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879761    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e93e581e-424f-4960-b2e9-fa94c29f0779" containerName="registry"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879768    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="thanos-sidecar"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879775    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="prometheus"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879781    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="config-reloader"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879787    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="kube-rbac-proxy"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879792    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="126d14d7-717b-4929-8774-b0575253ea5f" containerName="kube-rbac-proxy-web"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879814    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"} err="failed to get container status \"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4\": rpc error: code = NotFound desc = could not find container \"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4\": container with ID starting with eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4 not found: ID does not exist"
Sep 15 11:27:48.879856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.879827    2560 scope.go:117] "RemoveContainer" containerID="287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"
Sep 15 11:27:48.880478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.880007    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"} err="failed to get container status \"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990\": rpc error: code = NotFound desc = could not find container \"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990\": container with ID starting with 287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990 not found: ID does not exist"
Sep 15 11:27:48.880478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.880022    2560 scope.go:117] "RemoveContainer" containerID="d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"
Sep 15 11:27:48.880478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.880216    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"} err="failed to get container status \"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d\": rpc error: code = NotFound desc = could not find container \"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d\": container with ID starting with d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d not found: ID does not exist"
Sep 15 11:27:48.880478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.880228    2560 scope.go:117] "RemoveContainer" containerID="508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"
Sep 15 11:27:48.880478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.880405    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"} err="failed to get container status \"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8\": rpc error: code = NotFound desc = could not find container \"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8\": container with ID starting with 508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8 not found: ID does not exist"
Sep 15 11:27:48.880478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.880417    2560 scope.go:117] "RemoveContainer" containerID="da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09"
Sep 15 11:27:48.880742 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.880671    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09"} err="failed to get container status \"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09\": rpc error: code = NotFound desc = could not find container \"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09\": container with ID starting with da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09 not found: ID does not exist"
Sep 15 11:27:48.880742 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.880681    2560 scope.go:117] "RemoveContainer" containerID="a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"
Sep 15 11:27:48.880866 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.880849    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"} err="failed to get container status \"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4\": rpc error: code = NotFound desc = could not find container \"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4\": container with ID starting with a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4 not found: ID does not exist"
Sep 15 11:27:48.880908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.880868    2560 scope.go:117] "RemoveContainer" containerID="abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"
Sep 15 11:27:48.881052 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.881036    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"} err="failed to get container status \"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117\": rpc error: code = NotFound desc = could not find container \"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117\": container with ID starting with abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117 not found: ID does not exist"
Sep 15 11:27:48.881092 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.881052    2560 scope.go:117] "RemoveContainer" containerID="eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"
Sep 15 11:27:48.881241 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.881225    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"} err="failed to get container status \"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4\": rpc error: code = NotFound desc = could not find container \"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4\": container with ID starting with eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4 not found: ID does not exist"
Sep 15 11:27:48.881282 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.881241    2560 scope.go:117] "RemoveContainer" containerID="287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"
Sep 15 11:27:48.881442 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.881425    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"} err="failed to get container status \"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990\": rpc error: code = NotFound desc = could not find container \"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990\": container with ID starting with 287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990 not found: ID does not exist"
Sep 15 11:27:48.881487 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.881442    2560 scope.go:117] "RemoveContainer" containerID="d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"
Sep 15 11:27:48.881659 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.881644    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"} err="failed to get container status \"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d\": rpc error: code = NotFound desc = could not find container \"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d\": container with ID starting with d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d not found: ID does not exist"
Sep 15 11:27:48.881701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.881659    2560 scope.go:117] "RemoveContainer" containerID="508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"
Sep 15 11:27:48.881871 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.881845    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"} err="failed to get container status \"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8\": rpc error: code = NotFound desc = could not find container \"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8\": container with ID starting with 508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8 not found: ID does not exist"
Sep 15 11:27:48.881923 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.881873    2560 scope.go:117] "RemoveContainer" containerID="da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09"
Sep 15 11:27:48.882125 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.882105    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09"} err="failed to get container status \"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09\": rpc error: code = NotFound desc = could not find container \"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09\": container with ID starting with da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09 not found: ID does not exist"
Sep 15 11:27:48.882192 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.882127    2560 scope.go:117] "RemoveContainer" containerID="a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"
Sep 15 11:27:48.882377 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.882349    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4"} err="failed to get container status \"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4\": rpc error: code = NotFound desc = could not find container \"a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4\": container with ID starting with a85643ed8e95fde8cbb84cf771b66a1a42c62a821e9a5727ac53238c71268ce4 not found: ID does not exist"
Sep 15 11:27:48.882448 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.882379    2560 scope.go:117] "RemoveContainer" containerID="abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"
Sep 15 11:27:48.882606 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.882586    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117"} err="failed to get container status \"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117\": rpc error: code = NotFound desc = could not find container \"abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117\": container with ID starting with abdac170f23b889b46410a36644a2f063062c7e41473db3015132d07d01c3117 not found: ID does not exist"
Sep 15 11:27:48.882659 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.882607    2560 scope.go:117] "RemoveContainer" containerID="eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"
Sep 15 11:27:48.882822 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.882805    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4"} err="failed to get container status \"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4\": rpc error: code = NotFound desc = could not find container \"eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4\": container with ID starting with eb1b2d6f860cb09b456757501360fe5c042e9cab41048d281275d080286dbfc4 not found: ID does not exist"
Sep 15 11:27:48.882822 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.882822    2560 scope.go:117] "RemoveContainer" containerID="287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"
Sep 15 11:27:48.883057 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.883038    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990"} err="failed to get container status \"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990\": rpc error: code = NotFound desc = could not find container \"287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990\": container with ID starting with 287923f10849ea8a038a41e578e5d56a8ccb7233695315e927b1c9680dcd7990 not found: ID does not exist"
Sep 15 11:27:48.883105 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.883058    2560 scope.go:117] "RemoveContainer" containerID="d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"
Sep 15 11:27:48.883301 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.883283    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d"} err="failed to get container status \"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d\": rpc error: code = NotFound desc = could not find container \"d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d\": container with ID starting with d19928f6716fd9bd3a725e63327ce1b02bd63b7ee11c91d0623debbc92f0346d not found: ID does not exist"
Sep 15 11:27:48.883379 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.883303    2560 scope.go:117] "RemoveContainer" containerID="508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"
Sep 15 11:27:48.883544 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.883525    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8"} err="failed to get container status \"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8\": rpc error: code = NotFound desc = could not find container \"508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8\": container with ID starting with 508aa4d237c8c43a8bcb5b650880e59c119342e1bea3b9b4a82cb9a082c45fd8 not found: ID does not exist"
Sep 15 11:27:48.883614 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.883545    2560 scope.go:117] "RemoveContainer" containerID="da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09"
Sep 15 11:27:48.883783 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.883759    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09"} err="failed to get container status \"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09\": rpc error: code = NotFound desc = could not find container \"da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09\": container with ID starting with da3d8d4c5f127ca8902bb1178f9c3ea1d04fd923ab8c6d35b2f24420f46a5c09 not found: ID does not exist"
Sep 15 11:27:48.885107 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.885093    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.887096 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.887078    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 15 11:27:48.887178 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.887093    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 15 11:27:48.887609 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.887585    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 15 11:27:48.887710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.887660    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 15 11:27:48.887710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.887677    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 15 11:27:48.887710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.887683    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 15 11:27:48.887866 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.887703    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 15 11:27:48.887866 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.887792    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d208n1b14g9ge\""
Sep 15 11:27:48.887866 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.887852    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-cp4vc\""
Sep 15 11:27:48.887994 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.887971    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 15 11:27:48.888164 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.888149    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 15 11:27:48.888820 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.888806    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 15 11:27:48.888917 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.888903    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 15 11:27:48.889318 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.889303    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 15 11:27:48.889386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.889337    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 15 11:27:48.891157 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.891087    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 15 11:27:48.893237 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.893219    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 15 11:27:48.899989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.899970    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900079 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.899998    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900079 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900016    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900079 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900031    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900079 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900068    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900092    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900115    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2778a33d-a7b1-4dc9-82bc-31058249360d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900136    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl8r8\" (UniqueName: \"kubernetes.io/projected/2778a33d-a7b1-4dc9-82bc-31058249360d-kube-api-access-nl8r8\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900153    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900186    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900213    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900423 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900231    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-config\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900423 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900247    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900423 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900267    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900423 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900285    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2778a33d-a7b1-4dc9-82bc-31058249360d-config-out\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900423 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900300    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-web-config\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900423 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900317    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900423 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900336    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900423 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900351    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2778a33d-a7b1-4dc9-82bc-31058249360d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.900423 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.900365    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:48.904430 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:48.904408    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 11:27:49.001657 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.001623    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.001657 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.001661    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.001677    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.001699    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2778a33d-a7b1-4dc9-82bc-31058249360d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.001725    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl8r8\" (UniqueName: \"kubernetes.io/projected/2778a33d-a7b1-4dc9-82bc-31058249360d-kube-api-access-nl8r8\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.001750    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.001784    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.001811    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.001835    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-config\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.001865    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.001893    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.001922    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2778a33d-a7b1-4dc9-82bc-31058249360d-config-out\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.001945    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-web-config\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.001972    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.002003    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.002028    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2778a33d-a7b1-4dc9-82bc-31058249360d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.002050    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.002093    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.002136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.002127    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.003057 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.002137    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2778a33d-a7b1-4dc9-82bc-31058249360d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.003057 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.002309    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.003057 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.002428    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.003057 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.002760    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.003282 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.003256    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.004248 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.003907    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.005402 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.005053    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.005402 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.005236    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.005402 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.005342    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.005402 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.005355    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.005699 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.005402    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.006192 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.006149    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.006192 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.006166    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-config\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.006388 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.006364    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2778a33d-a7b1-4dc9-82bc-31058249360d-config-out\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.006787 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.006762    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.007439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.006995    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-web-config\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.007439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.007404    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.008267 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.008247    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2778a33d-a7b1-4dc9-82bc-31058249360d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.008454 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.008436    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2778a33d-a7b1-4dc9-82bc-31058249360d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.008670 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.008649    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2778a33d-a7b1-4dc9-82bc-31058249360d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.009664 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.009638    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl8r8\" (UniqueName: \"kubernetes.io/projected/2778a33d-a7b1-4dc9-82bc-31058249360d-kube-api-access-nl8r8\") pod \"prometheus-k8s-0\" (UID: \"2778a33d-a7b1-4dc9-82bc-31058249360d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.033445 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.033413    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="126d14d7-717b-4929-8774-b0575253ea5f" path="/var/lib/kubelet/pods/126d14d7-717b-4929-8774-b0575253ea5f/volumes"
Sep 15 11:27:49.194958 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.194919    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:49.480389 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.480317    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 15 11:27:49.483189 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:27:49.483159    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2778a33d_a7b1_4dc9_82bc_31058249360d.slice/crio-14920b7bbe9db8ddbc3e3e27c2096b6926fbbba0fde7db79b3f60960a2828abe WatchSource:0}: Error finding container 14920b7bbe9db8ddbc3e3e27c2096b6926fbbba0fde7db79b3f60960a2828abe: Status 404 returned error can't find the container with id 14920b7bbe9db8ddbc3e3e27c2096b6926fbbba0fde7db79b3f60960a2828abe
Sep 15 11:27:49.823776 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.823677    2560 generic.go:356] "Generic (PLEG): container finished" podID="2778a33d-a7b1-4dc9-82bc-31058249360d" containerID="a6c8877d50bb2b4266c771d770b063ebdde6d254abb07d019bfc42bb5e2524a5" exitCode=0
Sep 15 11:27:49.823776 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.823753    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2778a33d-a7b1-4dc9-82bc-31058249360d","Type":"ContainerDied","Data":"a6c8877d50bb2b4266c771d770b063ebdde6d254abb07d019bfc42bb5e2524a5"}
Sep 15 11:27:49.824586 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:49.824214    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2778a33d-a7b1-4dc9-82bc-31058249360d","Type":"ContainerStarted","Data":"14920b7bbe9db8ddbc3e3e27c2096b6926fbbba0fde7db79b3f60960a2828abe"}
Sep 15 11:27:50.110024 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:50.109999    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:27:50.838128 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:50.838095    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2778a33d-a7b1-4dc9-82bc-31058249360d","Type":"ContainerStarted","Data":"e42da1101cef7e9d5fb2c769c46302d93429ed7cc11fa5e800bc048f063b3705"}
Sep 15 11:27:50.838275 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:50.838158    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2778a33d-a7b1-4dc9-82bc-31058249360d","Type":"ContainerStarted","Data":"f121566190671652322f8806b27e99783901950998dd46be5ebd89d68e67d242"}
Sep 15 11:27:50.838275 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:50.838186    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2778a33d-a7b1-4dc9-82bc-31058249360d","Type":"ContainerStarted","Data":"45d2d6ac5def59ee67d1061352cbc98485d9988a7971239e0472e30a5595cbf7"}
Sep 15 11:27:50.838275 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:50.838211    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2778a33d-a7b1-4dc9-82bc-31058249360d","Type":"ContainerStarted","Data":"1f9a4d40671dab8dc2e4ed9f28c0422027c5e63dfc5410bfc6b6e60c9d7af072"}
Sep 15 11:27:50.838275 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:50.838233    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2778a33d-a7b1-4dc9-82bc-31058249360d","Type":"ContainerStarted","Data":"0ca357b736ef6fc766633d79f021d3afc4e34937618d456e8a3de609594ef4a7"}
Sep 15 11:27:50.838275 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:50.838256    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2778a33d-a7b1-4dc9-82bc-31058249360d","Type":"ContainerStarted","Data":"e6dc41775c4fbf299dd5bce2fe55635f1244257fae972068d302996effcd92dc"}
Sep 15 11:27:50.868126 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:50.868075    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.851300183 podStartE2EDuration="2.868060175s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.759992ms" isStatefulPod=false podCreationTimestamp="2026-09-15 11:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:27:50.865743968 +0000 UTC m=+172.358521119" watchObservedRunningTime="2026-09-15 11:27:50.868060175 +0000 UTC m=+172.360837322"
Sep 15 11:27:51.444356 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:51.444327    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:27:51.502715 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:51.502687    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:27:54.195436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:54.195402    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:27:57.775602 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:57.775547    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:27:57.839439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:57.839406    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:27:58.553213 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:58.553176    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:27:58.654057 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:58.654028    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:27:58.717426 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:27:58.717393    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:28:09.113416 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:28:09.113378    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:28:09.214293 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:28:09.214259    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:28:09.320569 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:28:09.320524    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:28:21.405648 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:28:21.405610    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:28:22.414262 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:28:22.414219    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:28:27.610425 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:28:27.610380    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:28:27.710899 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:28:27.710819    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:28:49.195297 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:28:49.195245    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:28:49.210529 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:28:49.210506    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:28:50.075964 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:28:50.075938    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 15 11:28:59.879470 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:28:59.879432    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:28:59.946076 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:28:59.946041    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:29:19.076077 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:19.076046    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:29:19.175290 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:19.175253    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:29:46.327290 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:46.327253    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:29:47.821877 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:47.821837    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:29:50.085193 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:50.085159    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:29:50.185939 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:50.185901    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:29:51.856262 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:51.856225    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:29:51.980212 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:51.980176    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:29:52.479622 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:52.479581    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:29:52.585264 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:52.585233    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:29:54.672235 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:54.672197    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:29:55.058760 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:55.058673    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:29:57.081704 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:57.081666    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:29:57.182336 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:57.182307    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:29:58.933506 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:58.933479    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:29:58.938596 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:58.938572    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:29:58.944790 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:29:58.944770    2560 kubelet.go:1648] "Image garbage collection succeeded"
Sep 15 11:30:02.678528 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:02.678485    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:05.986960 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:05.986926    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:09.365638 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:09.365598    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:09.473735 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:09.473704    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:11.011116 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:11.011081    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:26.731782 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:26.731735    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:26.835679 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:26.835639    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:27.313532 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.313497    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:27.625457 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.625423    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:27.734885 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.734851    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:27.766836 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.766805    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"]
Sep 15 11:30:27.770038 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.770021    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.773203 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.773176    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 15 11:30:27.773460 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.773442    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 15 11:30:27.773527 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.773462    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vtkfp\""
Sep 15 11:30:27.774434 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.774416    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 15 11:30:27.775034 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.775020    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 15 11:30:27.796024 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.795999    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"]
Sep 15 11:30:27.869598 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.869549    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7f79\" (UniqueName: \"kubernetes.io/projected/a9f74621-3355-4e42-a550-d7500eeec2d1-kube-api-access-r7f79\") pod \"opendatahub-operator-controller-manager-b96c7c888-n2l58\" (UID: \"a9f74621-3355-4e42-a550-d7500eeec2d1\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.869753 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.869606    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a9f74621-3355-4e42-a550-d7500eeec2d1-manifests\") pod \"opendatahub-operator-controller-manager-b96c7c888-n2l58\" (UID: \"a9f74621-3355-4e42-a550-d7500eeec2d1\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.869753 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.869657    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a9f74621-3355-4e42-a550-d7500eeec2d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b96c7c888-n2l58\" (UID: \"a9f74621-3355-4e42-a550-d7500eeec2d1\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.869753 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.869722    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9f74621-3355-4e42-a550-d7500eeec2d1-tmp\") pod \"opendatahub-operator-controller-manager-b96c7c888-n2l58\" (UID: \"a9f74621-3355-4e42-a550-d7500eeec2d1\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.869753 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.869741    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a9f74621-3355-4e42-a550-d7500eeec2d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-b96c7c888-n2l58\" (UID: \"a9f74621-3355-4e42-a550-d7500eeec2d1\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.970216 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.970133    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7f79\" (UniqueName: \"kubernetes.io/projected/a9f74621-3355-4e42-a550-d7500eeec2d1-kube-api-access-r7f79\") pod \"opendatahub-operator-controller-manager-b96c7c888-n2l58\" (UID: \"a9f74621-3355-4e42-a550-d7500eeec2d1\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.970216 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.970170    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a9f74621-3355-4e42-a550-d7500eeec2d1-manifests\") pod \"opendatahub-operator-controller-manager-b96c7c888-n2l58\" (UID: \"a9f74621-3355-4e42-a550-d7500eeec2d1\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.970417 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.970198    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a9f74621-3355-4e42-a550-d7500eeec2d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b96c7c888-n2l58\" (UID: \"a9f74621-3355-4e42-a550-d7500eeec2d1\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.970417 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.970336    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9f74621-3355-4e42-a550-d7500eeec2d1-tmp\") pod \"opendatahub-operator-controller-manager-b96c7c888-n2l58\" (UID: \"a9f74621-3355-4e42-a550-d7500eeec2d1\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.970417 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.970363    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a9f74621-3355-4e42-a550-d7500eeec2d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-b96c7c888-n2l58\" (UID: \"a9f74621-3355-4e42-a550-d7500eeec2d1\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.970680 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.970658    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a9f74621-3355-4e42-a550-d7500eeec2d1-manifests\") pod \"opendatahub-operator-controller-manager-b96c7c888-n2l58\" (UID: \"a9f74621-3355-4e42-a550-d7500eeec2d1\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.972585 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.972535    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9f74621-3355-4e42-a550-d7500eeec2d1-tmp\") pod \"opendatahub-operator-controller-manager-b96c7c888-n2l58\" (UID: \"a9f74621-3355-4e42-a550-d7500eeec2d1\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.972720 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.972697    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a9f74621-3355-4e42-a550-d7500eeec2d1-webhook-cert\") pod \"opendatahub-operator-controller-manager-b96c7c888-n2l58\" (UID: \"a9f74621-3355-4e42-a550-d7500eeec2d1\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.972841 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.972827    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a9f74621-3355-4e42-a550-d7500eeec2d1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b96c7c888-n2l58\" (UID: \"a9f74621-3355-4e42-a550-d7500eeec2d1\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.981386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.981356    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7f79\" (UniqueName: \"kubernetes.io/projected/a9f74621-3355-4e42-a550-d7500eeec2d1-kube-api-access-r7f79\") pod \"opendatahub-operator-controller-manager-b96c7c888-n2l58\" (UID: \"a9f74621-3355-4e42-a550-d7500eeec2d1\") " pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:27.995470 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:27.995448    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:28.080102 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:28.080062    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:28.157843 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:28.157804    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:28.252156 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:28.252076    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:28.354798 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:28.354759    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:28.369995 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:28.369970    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"]
Sep 15 11:30:28.373605 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:30:28.373580    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9f74621_3355_4e42_a550_d7500eeec2d1.slice/crio-ca31f2bac9a49c017f9d06f0be9c5e79b115ccca8c777ac66fd345612ed347b6 WatchSource:0}: Error finding container ca31f2bac9a49c017f9d06f0be9c5e79b115ccca8c777ac66fd345612ed347b6: Status 404 returned error can't find the container with id ca31f2bac9a49c017f9d06f0be9c5e79b115ccca8c777ac66fd345612ed347b6
Sep 15 11:30:28.380167 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:28.376172    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 11:30:28.416120 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:28.416086    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58" event={"ID":"a9f74621-3355-4e42-a550-d7500eeec2d1","Type":"ContainerStarted","Data":"ca31f2bac9a49c017f9d06f0be9c5e79b115ccca8c777ac66fd345612ed347b6"}
Sep 15 11:30:28.450490 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:28.450459    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:28.553648 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:28.553578    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:28.659838 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:28.659806    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:28.776412 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:28.776385    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:31.430442 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:31.430411    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58" event={"ID":"a9f74621-3355-4e42-a550-d7500eeec2d1","Type":"ContainerStarted","Data":"49c173df9d73e536760d0fa1f03384d10c31a174e09b1b3c7ce7034da494a244"}
Sep 15 11:30:31.603357 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:31.603317    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:31.718422 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:31.718382    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:33.440522 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:33.440490    2560 generic.go:356] "Generic (PLEG): container finished" podID="a9f74621-3355-4e42-a550-d7500eeec2d1" containerID="49c173df9d73e536760d0fa1f03384d10c31a174e09b1b3c7ce7034da494a244" exitCode=0
Sep 15 11:30:33.440981 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:33.440536    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58" event={"ID":"a9f74621-3355-4e42-a550-d7500eeec2d1","Type":"ContainerDied","Data":"49c173df9d73e536760d0fa1f03384d10c31a174e09b1b3c7ce7034da494a244"}
Sep 15 11:30:34.446659 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:34.446622    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58" event={"ID":"a9f74621-3355-4e42-a550-d7500eeec2d1","Type":"ContainerStarted","Data":"c88bdfe002539547d396a8815bb16f75c5a6d438c5474cfbca8a715fbb3c2b8f"}
Sep 15 11:30:34.447037 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:34.446823    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:34.476348 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:34.476303    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58" podStartSLOduration=3.04997598 podStartE2EDuration="7.476288957s" totalImagesPullingTime="3.336953475s" totalInitContainerRuntime="1.089359502s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:30:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:30:34.472248793 +0000 UTC m=+335.965025942" watchObservedRunningTime="2026-09-15 11:30:34.476288957 +0000 UTC m=+335.969066104"
Sep 15 11:30:37.287267 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:37.287230    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:37.396220 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:37.396184    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:45.452690 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:45.452659    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b96c7c888-n2l58"
Sep 15 11:30:47.589642 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:47.589609    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:47.791181 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:47.791142    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:48.188419 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.188379    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:48.228966 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.228939    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-79674675df-hkgzr"]
Sep 15 11:30:48.241705 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.241681    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79674675df-hkgzr"
Sep 15 11:30:48.251928 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.251908    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sdl7t\""
Sep 15 11:30:48.253633 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.253601    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 15 11:30:48.253956 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.253942    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 15 11:30:48.274165 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.274143    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79674675df-hkgzr"]
Sep 15 11:30:48.305312 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.305291    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/feaf279e-bf6c-4cfd-9778-fccc641b7150-tmp\") pod \"kube-auth-proxy-79674675df-hkgzr\" (UID: \"feaf279e-bf6c-4cfd-9778-fccc641b7150\") " pod="openshift-ingress/kube-auth-proxy-79674675df-hkgzr"
Sep 15 11:30:48.305427 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.305320    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/feaf279e-bf6c-4cfd-9778-fccc641b7150-tls-certs\") pod \"kube-auth-proxy-79674675df-hkgzr\" (UID: \"feaf279e-bf6c-4cfd-9778-fccc641b7150\") " pod="openshift-ingress/kube-auth-proxy-79674675df-hkgzr"
Sep 15 11:30:48.305427 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.305341    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whrv2\" (UniqueName: \"kubernetes.io/projected/feaf279e-bf6c-4cfd-9778-fccc641b7150-kube-api-access-whrv2\") pod \"kube-auth-proxy-79674675df-hkgzr\" (UID: \"feaf279e-bf6c-4cfd-9778-fccc641b7150\") " pod="openshift-ingress/kube-auth-proxy-79674675df-hkgzr"
Sep 15 11:30:48.405848 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.405819    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/feaf279e-bf6c-4cfd-9778-fccc641b7150-tmp\") pod \"kube-auth-proxy-79674675df-hkgzr\" (UID: \"feaf279e-bf6c-4cfd-9778-fccc641b7150\") " pod="openshift-ingress/kube-auth-proxy-79674675df-hkgzr"
Sep 15 11:30:48.405848 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.405852    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/feaf279e-bf6c-4cfd-9778-fccc641b7150-tls-certs\") pod \"kube-auth-proxy-79674675df-hkgzr\" (UID: \"feaf279e-bf6c-4cfd-9778-fccc641b7150\") " pod="openshift-ingress/kube-auth-proxy-79674675df-hkgzr"
Sep 15 11:30:48.406050 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.405881    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whrv2\" (UniqueName: \"kubernetes.io/projected/feaf279e-bf6c-4cfd-9778-fccc641b7150-kube-api-access-whrv2\") pod \"kube-auth-proxy-79674675df-hkgzr\" (UID: \"feaf279e-bf6c-4cfd-9778-fccc641b7150\") " pod="openshift-ingress/kube-auth-proxy-79674675df-hkgzr"
Sep 15 11:30:48.408201 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.408170    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/feaf279e-bf6c-4cfd-9778-fccc641b7150-tmp\") pod \"kube-auth-proxy-79674675df-hkgzr\" (UID: \"feaf279e-bf6c-4cfd-9778-fccc641b7150\") " pod="openshift-ingress/kube-auth-proxy-79674675df-hkgzr"
Sep 15 11:30:48.408302 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.408283    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/feaf279e-bf6c-4cfd-9778-fccc641b7150-tls-certs\") pod \"kube-auth-proxy-79674675df-hkgzr\" (UID: \"feaf279e-bf6c-4cfd-9778-fccc641b7150\") " pod="openshift-ingress/kube-auth-proxy-79674675df-hkgzr"
Sep 15 11:30:48.460394 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.460335    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whrv2\" (UniqueName: \"kubernetes.io/projected/feaf279e-bf6c-4cfd-9778-fccc641b7150-kube-api-access-whrv2\") pod \"kube-auth-proxy-79674675df-hkgzr\" (UID: \"feaf279e-bf6c-4cfd-9778-fccc641b7150\") " pod="openshift-ingress/kube-auth-proxy-79674675df-hkgzr"
Sep 15 11:30:48.481441 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.481403    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:48.550833 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.550807    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79674675df-hkgzr"
Sep 15 11:30:48.841702 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:48.841676    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79674675df-hkgzr"]
Sep 15 11:30:48.844692 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:30:48.844665    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeaf279e_bf6c_4cfd_9778_fccc641b7150.slice/crio-7b83b00d424aadeec3d0adfdfce9a9a9729351560362ed60a1e114694f2d50a5 WatchSource:0}: Error finding container 7b83b00d424aadeec3d0adfdfce9a9a9729351560362ed60a1e114694f2d50a5: Status 404 returned error can't find the container with id 7b83b00d424aadeec3d0adfdfce9a9a9729351560362ed60a1e114694f2d50a5
Sep 15 11:30:49.501064 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:49.501031    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79674675df-hkgzr" event={"ID":"feaf279e-bf6c-4cfd-9778-fccc641b7150","Type":"ContainerStarted","Data":"7b83b00d424aadeec3d0adfdfce9a9a9729351560362ed60a1e114694f2d50a5"}
Sep 15 11:30:51.633533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.633440    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:51.879944 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.879906    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"]
Sep 15 11:30:51.883062 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.883045    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:51.888812 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.888752    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 15 11:30:51.889766 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.889747    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 15 11:30:51.892070 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.892052    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 15 11:30:51.894985 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.894966    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 15 11:30:51.937701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.937675    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:51.937701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.937701    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:51.937872 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.937719    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:51.937872 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.937768    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:51.937872 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.937847    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f96mf\" (UniqueName: \"kubernetes.io/projected/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-kube-api-access-f96mf\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:51.938027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.937891    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:51.938027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.937906    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-nmg9l\""
Sep 15 11:30:51.938027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.937920    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:51.956942 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.956911    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"]
Sep 15 11:30:51.993189 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:51.993165    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:52.039272 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.039248    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:52.039360 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.039280    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:52.039360 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.039302    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:52.039360 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.039331    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:52.039360 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.039353    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:52.039517 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.039382    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:52.039517 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.039439    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f96mf\" (UniqueName: \"kubernetes.io/projected/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-kube-api-access-f96mf\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:52.040008 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.039985    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:52.041739 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.041720    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:52.041837 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.041819    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:52.042019 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.041999    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:52.042058 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.042006    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:52.077030 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.077006    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:52.079854 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.079830    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f96mf\" (UniqueName: \"kubernetes.io/projected/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-kube-api-access-f96mf\") pod \"istiod-openshift-gateway-7b7995d5cd-rjh54\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:52.192132 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.192072    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:52.497969 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.497944    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"]
Sep 15 11:30:52.499580 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:30:52.499542    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff1ffd3f_4c3b_4a3a_ad75_88a5cf3e1514.slice/crio-f8e4ec1a0d82a570420a94f62bab8d9a0138986ab95d2305c1cfb1288518983c WatchSource:0}: Error finding container f8e4ec1a0d82a570420a94f62bab8d9a0138986ab95d2305c1cfb1288518983c: Status 404 returned error can't find the container with id f8e4ec1a0d82a570420a94f62bab8d9a0138986ab95d2305c1cfb1288518983c
Sep 15 11:30:52.518455 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.518423    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79674675df-hkgzr" event={"ID":"feaf279e-bf6c-4cfd-9778-fccc641b7150","Type":"ContainerStarted","Data":"0d1a9f9e4a18d401dc1215a460acc8a4a326dc6a39ca6a4a630420c5ba6672f0"}
Sep 15 11:30:52.519629 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.519605    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54" event={"ID":"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514","Type":"ContainerStarted","Data":"f8e4ec1a0d82a570420a94f62bab8d9a0138986ab95d2305c1cfb1288518983c"}
Sep 15 11:30:52.553060 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:52.553006    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-79674675df-hkgzr" podStartSLOduration=1.687977589 podStartE2EDuration="4.552993227s" totalImagesPullingTime="2.865015638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:30:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:30:52.55161394 +0000 UTC m=+354.044391087" watchObservedRunningTime="2026-09-15 11:30:52.552993227 +0000 UTC m=+354.045770372"
Sep 15 11:30:54.027263 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:54.027221    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:56.315949 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:56.315907    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:30:56.316239 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:56.315978    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:30:56.538759 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:56.538725    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54" event={"ID":"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514","Type":"ContainerStarted","Data":"a77f603a3b2109f4de022341866a516c3b74aa31be1190a2a79ce3cb5ffd0486"}
Sep 15 11:30:56.538950 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:56.538847    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:30:56.560253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:56.560200    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54" podStartSLOduration=1.746133986 podStartE2EDuration="5.560185875s" totalImagesPullingTime="3.814051889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:30:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:30:56.558135313 +0000 UTC m=+358.050912460" watchObservedRunningTime="2026-09-15 11:30:56.560185875 +0000 UTC m=+358.052963022"
Sep 15 11:30:57.186061 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:57.186029    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:57.266377 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:57.266329    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:57.544535 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:57.544445    2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-rjh54 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 11:30:57.544535 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:57.544498    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54" podUID="ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 11:30:57.857552 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:57.857515    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:58.090159 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:58.090114    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:58.236184 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:58.236098    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:58.484584 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:58.484531    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:59.113552 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:59.113513    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:59.217423 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:59.217390    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:30:59.861289 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:30:59.861256    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:00.544214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:00.544184    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:31:06.110913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.110878    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"]
Sep 15 11:31:06.115709 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.115692    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.129010 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.128985    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"]
Sep 15 11:31:06.260979 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.260951    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfztn\" (UniqueName: \"kubernetes.io/projected/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-kube-api-access-lfztn\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.261109 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.261003    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.261109 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.261071    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.261185 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.261116    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.261185 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.261142    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.261185 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.261157    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.261185 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.261182    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.362439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.362367    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.362439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.362406    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfztn\" (UniqueName: \"kubernetes.io/projected/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-kube-api-access-lfztn\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.362679 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.362535    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.362679 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.362622    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.362679 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.362666    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.362829 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.362703    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.362829 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.362729    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.363388 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.363362    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.364842 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.364820    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.364920 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.364893    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.364971 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.364946    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.364971 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.364964    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.389904 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.389873    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.395095 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.395065    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfztn\" (UniqueName: \"kubernetes.io/projected/37b6d3ea-fd8f-4958-8246-4ef0aced2bb8-kube-api-access-lfztn\") pod \"istiod-openshift-gateway-5677ccb49b-pjn7w\" (UID: \"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.424169 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.424129    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:06.709194 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.708988    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"]
Sep 15 11:31:06.711613 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:31:06.711586    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37b6d3ea_fd8f_4958_8246_4ef0aced2bb8.slice/crio-dd845c0f3ab14cbf22e260cf82e77f0d5bc2720d7ab991f428af01fa060dc281 WatchSource:0}: Error finding container dd845c0f3ab14cbf22e260cf82e77f0d5bc2720d7ab991f428af01fa060dc281: Status 404 returned error can't find the container with id dd845c0f3ab14cbf22e260cf82e77f0d5bc2720d7ab991f428af01fa060dc281
Sep 15 11:31:06.713715 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.713686    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:31:06.713774 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:06.713745    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:31:07.587087 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:07.587043    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w" event={"ID":"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8","Type":"ContainerStarted","Data":"5d0410ed2e9d05ccb6042ac061154c858d29d0979120e7a94674f7240c007a20"}
Sep 15 11:31:07.587505 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:07.587096    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w" event={"ID":"37b6d3ea-fd8f-4958-8246-4ef0aced2bb8","Type":"ContainerStarted","Data":"dd845c0f3ab14cbf22e260cf82e77f0d5bc2720d7ab991f428af01fa060dc281"}
Sep 15 11:31:07.587505 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:07.587214    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:07.588768 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:07.588747    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w"
Sep 15 11:31:07.651388 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:07.651335    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-pjn7w" podStartSLOduration=1.651318292 podStartE2EDuration="1.651318292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:31:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:31:07.64909537 +0000 UTC m=+369.141872519" watchObservedRunningTime="2026-09-15 11:31:07.651318292 +0000 UTC m=+369.144095440"
Sep 15 11:31:07.736209 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:07.736180    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"]
Sep 15 11:31:07.736390 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:07.736354    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54" podUID="ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514" containerName="discovery" containerID="cri-o://a77f603a3b2109f4de022341866a516c3b74aa31be1190a2a79ce3cb5ffd0486" gracePeriod=30
Sep 15 11:31:07.969164 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:07.969141    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:31:08.077185 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.077152    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-local-certs\") pod \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") "
Sep 15 11:31:08.077185 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.077186    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-cacerts\") pod \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") "
Sep 15 11:31:08.077414 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.077218    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-token\") pod \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") "
Sep 15 11:31:08.077414 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.077272    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-kubeconfig\") pod \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") "
Sep 15 11:31:08.077414 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.077317    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-csr-ca-configmap\") pod \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") "
Sep 15 11:31:08.077414 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.077349    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-csr-dns-cert\") pod \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") "
Sep 15 11:31:08.077414 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.077385    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f96mf\" (UniqueName: \"kubernetes.io/projected/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-kube-api-access-f96mf\") pod \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\" (UID: \"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514\") "
Sep 15 11:31:08.077746 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.077714    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514" (UID: "ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:31:08.079623 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.079589    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-token" (OuterVolumeSpecName: "istio-token") pod "ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514" (UID: "ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:31:08.079729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.079673    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-local-certs" (OuterVolumeSpecName: "local-certs") pod "ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514" (UID: "ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:31:08.079793 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.079732    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-cacerts" (OuterVolumeSpecName: "cacerts") pod "ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514" (UID: "ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:31:08.079793 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.079744    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514" (UID: "ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:31:08.079971 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.079949    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-kube-api-access-f96mf" (OuterVolumeSpecName: "kube-api-access-f96mf") pod "ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514" (UID: "ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514"). InnerVolumeSpecName "kube-api-access-f96mf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:31:08.080078 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.080057    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514" (UID: "ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:31:08.178900 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.178846    2560 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-kubeconfig\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:31:08.178900 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.178868    2560 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-csr-ca-configmap\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:31:08.178900 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.178877    2560 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-csr-dns-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:31:08.178900 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.178887    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f96mf\" (UniqueName: \"kubernetes.io/projected/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-kube-api-access-f96mf\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:31:08.178900 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.178896    2560 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-local-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:31:08.179106 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.178904    2560 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-cacerts\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:31:08.179106 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.178912    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:31:08.592821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.592787    2560 generic.go:356] "Generic (PLEG): container finished" podID="ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514" containerID="a77f603a3b2109f4de022341866a516c3b74aa31be1190a2a79ce3cb5ffd0486" exitCode=0
Sep 15 11:31:08.593233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.592826    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"
Sep 15 11:31:08.593233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.592874    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54" event={"ID":"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514","Type":"ContainerDied","Data":"a77f603a3b2109f4de022341866a516c3b74aa31be1190a2a79ce3cb5ffd0486"}
Sep 15 11:31:08.593233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.592920    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54" event={"ID":"ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514","Type":"ContainerDied","Data":"f8e4ec1a0d82a570420a94f62bab8d9a0138986ab95d2305c1cfb1288518983c"}
Sep 15 11:31:08.593233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.592946    2560 scope.go:117] "RemoveContainer" containerID="a77f603a3b2109f4de022341866a516c3b74aa31be1190a2a79ce3cb5ffd0486"
Sep 15 11:31:08.602048 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.602033    2560 scope.go:117] "RemoveContainer" containerID="a77f603a3b2109f4de022341866a516c3b74aa31be1190a2a79ce3cb5ffd0486"
Sep 15 11:31:08.602308 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:31:08.602289    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a77f603a3b2109f4de022341866a516c3b74aa31be1190a2a79ce3cb5ffd0486\": container with ID starting with a77f603a3b2109f4de022341866a516c3b74aa31be1190a2a79ce3cb5ffd0486 not found: ID does not exist" containerID="a77f603a3b2109f4de022341866a516c3b74aa31be1190a2a79ce3cb5ffd0486"
Sep 15 11:31:08.602375 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.602316    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a77f603a3b2109f4de022341866a516c3b74aa31be1190a2a79ce3cb5ffd0486"} err="failed to get container status \"a77f603a3b2109f4de022341866a516c3b74aa31be1190a2a79ce3cb5ffd0486\": rpc error: code = NotFound desc = could not find container \"a77f603a3b2109f4de022341866a516c3b74aa31be1190a2a79ce3cb5ffd0486\": container with ID starting with a77f603a3b2109f4de022341866a516c3b74aa31be1190a2a79ce3cb5ffd0486 not found: ID does not exist"
Sep 15 11:31:08.643284 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.643262    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"]
Sep 15 11:31:08.646845 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:08.646827    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rjh54"]
Sep 15 11:31:09.033635 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:09.033539    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514" path="/var/lib/kubelet/pods/ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514/volumes"
Sep 15 11:31:09.871531 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:09.871499    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:10.006368 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.006329    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:10.034705 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.034677    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc"]
Sep 15 11:31:10.034977 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.034965    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514" containerName="discovery"
Sep 15 11:31:10.035022 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.034978    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514" containerName="discovery"
Sep 15 11:31:10.035068 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.035028    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff1ffd3f-4c3b-4a3a-ad75-88a5cf3e1514" containerName="discovery"
Sep 15 11:31:10.039254 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.039231    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc"
Sep 15 11:31:10.042914 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.042890    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 15 11:31:10.043045 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.042938    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-jv844\""
Sep 15 11:31:10.074739 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.074712    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc"]
Sep 15 11:31:10.120721 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.120688    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:10.196657 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.196595    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfl2c\" (UniqueName: \"kubernetes.io/projected/dd5db2e0-f577-40eb-8ec9-7af6169ef256-kube-api-access-bfl2c\") pod \"llmisvc-controller-manager-777c6bd7d5-kd2kc\" (UID: \"dd5db2e0-f577-40eb-8ec9-7af6169ef256\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc"
Sep 15 11:31:10.196657 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.196627    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd5db2e0-f577-40eb-8ec9-7af6169ef256-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kd2kc\" (UID: \"dd5db2e0-f577-40eb-8ec9-7af6169ef256\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc"
Sep 15 11:31:10.256107 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.256078    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:10.297922 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.297893    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfl2c\" (UniqueName: \"kubernetes.io/projected/dd5db2e0-f577-40eb-8ec9-7af6169ef256-kube-api-access-bfl2c\") pod \"llmisvc-controller-manager-777c6bd7d5-kd2kc\" (UID: \"dd5db2e0-f577-40eb-8ec9-7af6169ef256\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc"
Sep 15 11:31:10.298033 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.297923    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd5db2e0-f577-40eb-8ec9-7af6169ef256-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kd2kc\" (UID: \"dd5db2e0-f577-40eb-8ec9-7af6169ef256\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc"
Sep 15 11:31:10.300324 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.300301    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dd5db2e0-f577-40eb-8ec9-7af6169ef256-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kd2kc\" (UID: \"dd5db2e0-f577-40eb-8ec9-7af6169ef256\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc"
Sep 15 11:31:10.308452 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.308430    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfl2c\" (UniqueName: \"kubernetes.io/projected/dd5db2e0-f577-40eb-8ec9-7af6169ef256-kube-api-access-bfl2c\") pod \"llmisvc-controller-manager-777c6bd7d5-kd2kc\" (UID: \"dd5db2e0-f577-40eb-8ec9-7af6169ef256\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc"
Sep 15 11:31:10.349216 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.349191    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc"
Sep 15 11:31:10.374749 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.374724    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:10.494108 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.494033    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:10.633091 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.633060    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc"]
Sep 15 11:31:10.636174 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:31:10.636144    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddd5db2e0_f577_40eb_8ec9_7af6169ef256.slice/crio-da122fed0207c09517bcfab9705e983bb6415ce5f31918d2c1bad38719c72dc0 WatchSource:0}: Error finding container da122fed0207c09517bcfab9705e983bb6415ce5f31918d2c1bad38719c72dc0: Status 404 returned error can't find the container with id da122fed0207c09517bcfab9705e983bb6415ce5f31918d2c1bad38719c72dc0
Sep 15 11:31:10.724797 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:10.724762    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:11.613397 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:11.613365    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc" event={"ID":"dd5db2e0-f577-40eb-8ec9-7af6169ef256","Type":"ContainerStarted","Data":"da122fed0207c09517bcfab9705e983bb6415ce5f31918d2c1bad38719c72dc0"}
Sep 15 11:31:13.624793 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:13.624766    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc" event={"ID":"dd5db2e0-f577-40eb-8ec9-7af6169ef256","Type":"ContainerStarted","Data":"29d944e2e0d5df8b85e2b603e9973df6766b2b72027d1b1cd6042ec00dc473fc"}
Sep 15 11:31:13.625089 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:13.624896    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc"
Sep 15 11:31:13.658957 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:13.658915    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc" podStartSLOduration=1.801113594 podStartE2EDuration="4.658901866s" totalImagesPullingTime="2.857788272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:31:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:31:13.657140379 +0000 UTC m=+375.149917524" watchObservedRunningTime="2026-09-15 11:31:13.658901866 +0000 UTC m=+375.151679014"
Sep 15 11:31:17.450190 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:17.450144    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:17.526372 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:17.526328    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:28.145751 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:28.145714    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:28.246806 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:28.246769    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:28.302478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:28.302444    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:34.033728 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:34.033691    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:43.334926 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:43.334888    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:43.396475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:43.396435    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:44.631335 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:44.631303    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kd2kc"
Sep 15 11:31:59.609868 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:59.609832    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:31:59.703959 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:31:59.703928    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:13.410970 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:13.410932    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:15.491033 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:15.490995    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-lf6rx"]
Sep 15 11:32:15.495793 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:15.495767    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-lf6rx"
Sep 15 11:32:15.499768 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:15.499741    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 15 11:32:15.499768 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:15.499764    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qnwj5\""
Sep 15 11:32:15.500119 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:15.500105    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 15 11:32:15.500422 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:15.500404    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 15 11:32:15.512014 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:15.511987    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-lf6rx"]
Sep 15 11:32:15.617985 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:15.617953    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk6tx\" (UniqueName: \"kubernetes.io/projected/dc1335fa-fa3b-4fce-9e0f-4bb61c0dabe7-kube-api-access-hk6tx\") pod \"dns-operator-controller-manager-7b84cd7c9c-lf6rx\" (UID: \"dc1335fa-fa3b-4fce-9e0f-4bb61c0dabe7\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-lf6rx"
Sep 15 11:32:15.719051 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:15.719017    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk6tx\" (UniqueName: \"kubernetes.io/projected/dc1335fa-fa3b-4fce-9e0f-4bb61c0dabe7-kube-api-access-hk6tx\") pod \"dns-operator-controller-manager-7b84cd7c9c-lf6rx\" (UID: \"dc1335fa-fa3b-4fce-9e0f-4bb61c0dabe7\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-lf6rx"
Sep 15 11:32:15.754555 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:15.754495    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk6tx\" (UniqueName: \"kubernetes.io/projected/dc1335fa-fa3b-4fce-9e0f-4bb61c0dabe7-kube-api-access-hk6tx\") pod \"dns-operator-controller-manager-7b84cd7c9c-lf6rx\" (UID: \"dc1335fa-fa3b-4fce-9e0f-4bb61c0dabe7\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-lf6rx"
Sep 15 11:32:15.806236 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:15.806212    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-lf6rx"
Sep 15 11:32:16.043063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:16.042984    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:16.086219 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:16.086195    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-lf6rx"]
Sep 15 11:32:16.088736 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:32:16.088707    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc1335fa_fa3b_4fce_9e0f_4bb61c0dabe7.slice/crio-f44ba4350e3a77a58a4d1a782b8e2e7f3702b04034ea360e11fb5b3d4cafeb3c WatchSource:0}: Error finding container f44ba4350e3a77a58a4d1a782b8e2e7f3702b04034ea360e11fb5b3d4cafeb3c: Status 404 returned error can't find the container with id f44ba4350e3a77a58a4d1a782b8e2e7f3702b04034ea360e11fb5b3d4cafeb3c
Sep 15 11:32:16.404270 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:16.404239    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:16.820292 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:16.820210    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:16.893247 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:16.893212    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-lf6rx" event={"ID":"dc1335fa-fa3b-4fce-9e0f-4bb61c0dabe7","Type":"ContainerStarted","Data":"f44ba4350e3a77a58a4d1a782b8e2e7f3702b04034ea360e11fb5b3d4cafeb3c"}
Sep 15 11:32:18.068315 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:18.068276    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:19.805283 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:19.805251    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"]
Sep 15 11:32:19.809148 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:19.809122    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:19.815121 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:19.815097    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qslhn\""
Sep 15 11:32:19.836951 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:19.836923    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"]
Sep 15 11:32:19.886276 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:19.886252    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:19.959279 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:19.959245    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgs8l\" (UniqueName: \"kubernetes.io/projected/c401d308-5a6e-45fa-be66-f532e52bc884-kube-api-access-pgs8l\") pod \"kuadrant-operator-controller-manager-76f8f686bf-kh9kj\" (UID: \"c401d308-5a6e-45fa-be66-f532e52bc884\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:19.959412 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:19.959360    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c401d308-5a6e-45fa-be66-f532e52bc884-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-kh9kj\" (UID: \"c401d308-5a6e-45fa-be66-f532e52bc884\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:19.989389 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:19.989357    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:20.060002 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:20.059925    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c401d308-5a6e-45fa-be66-f532e52bc884-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-kh9kj\" (UID: \"c401d308-5a6e-45fa-be66-f532e52bc884\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:20.060002 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:20.059994    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgs8l\" (UniqueName: \"kubernetes.io/projected/c401d308-5a6e-45fa-be66-f532e52bc884-kube-api-access-pgs8l\") pod \"kuadrant-operator-controller-manager-76f8f686bf-kh9kj\" (UID: \"c401d308-5a6e-45fa-be66-f532e52bc884\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:20.060341 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:20.060317    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c401d308-5a6e-45fa-be66-f532e52bc884-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-kh9kj\" (UID: \"c401d308-5a6e-45fa-be66-f532e52bc884\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:20.071780 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:20.071748    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgs8l\" (UniqueName: \"kubernetes.io/projected/c401d308-5a6e-45fa-be66-f532e52bc884-kube-api-access-pgs8l\") pod \"kuadrant-operator-controller-manager-76f8f686bf-kh9kj\" (UID: \"c401d308-5a6e-45fa-be66-f532e52bc884\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:20.093171 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:20.093139    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:20.120459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:20.120432    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:20.395181 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:20.395148    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:20.403585 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:20.403547    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"]
Sep 15 11:32:20.407954 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:32:20.407925    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc401d308_5a6e_45fa_be66_f532e52bc884.slice/crio-7e2a7028e99b8929e59cd0bb0a11a316cbfa449ad5a67559fb044e1974fabc32 WatchSource:0}: Error finding container 7e2a7028e99b8929e59cd0bb0a11a316cbfa449ad5a67559fb044e1974fabc32: Status 404 returned error can't find the container with id 7e2a7028e99b8929e59cd0bb0a11a316cbfa449ad5a67559fb044e1974fabc32
Sep 15 11:32:20.912686 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:20.912652    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-lf6rx" event={"ID":"dc1335fa-fa3b-4fce-9e0f-4bb61c0dabe7","Type":"ContainerStarted","Data":"a4fb34107b3c507de7078c713868ff257c6fd87be2d2a40c99b48443eb780b3c"}
Sep 15 11:32:20.913134 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:20.912900    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-lf6rx"
Sep 15 11:32:20.913983 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:20.913963    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj" event={"ID":"c401d308-5a6e-45fa-be66-f532e52bc884","Type":"ContainerStarted","Data":"7e2a7028e99b8929e59cd0bb0a11a316cbfa449ad5a67559fb044e1974fabc32"}
Sep 15 11:32:20.930604 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:20.930540    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-lf6rx" podStartSLOduration=2.157936595 podStartE2EDuration="5.930527026s" totalImagesPullingTime="3.772590431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:32:20.93012054 +0000 UTC m=+442.422897690" watchObservedRunningTime="2026-09-15 11:32:20.930527026 +0000 UTC m=+442.423304170"
Sep 15 11:32:21.465081 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:21.465047    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:21.568948 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:21.568914    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:22.385917 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:22.385880    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-lg4gt"]
Sep 15 11:32:22.389608 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:22.389579    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-lg4gt"
Sep 15 11:32:22.391895 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:22.391871    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-zpxbk\""
Sep 15 11:32:22.402601 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:22.402574    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-lg4gt"]
Sep 15 11:32:22.477438 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:22.477397    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwqdx\" (UniqueName: \"kubernetes.io/projected/bba73fe3-5ba7-4c50-bb0e-7e5efe7b2c5a-kube-api-access-kwqdx\") pod \"authorino-operator-76b478bd78-lg4gt\" (UID: \"bba73fe3-5ba7-4c50-bb0e-7e5efe7b2c5a\") " pod="kuadrant-system/authorino-operator-76b478bd78-lg4gt"
Sep 15 11:32:22.578791 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:22.578740    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwqdx\" (UniqueName: \"kubernetes.io/projected/bba73fe3-5ba7-4c50-bb0e-7e5efe7b2c5a-kube-api-access-kwqdx\") pod \"authorino-operator-76b478bd78-lg4gt\" (UID: \"bba73fe3-5ba7-4c50-bb0e-7e5efe7b2c5a\") " pod="kuadrant-system/authorino-operator-76b478bd78-lg4gt"
Sep 15 11:32:22.594496 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:22.594465    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwqdx\" (UniqueName: \"kubernetes.io/projected/bba73fe3-5ba7-4c50-bb0e-7e5efe7b2c5a-kube-api-access-kwqdx\") pod \"authorino-operator-76b478bd78-lg4gt\" (UID: \"bba73fe3-5ba7-4c50-bb0e-7e5efe7b2c5a\") " pod="kuadrant-system/authorino-operator-76b478bd78-lg4gt"
Sep 15 11:32:22.703182 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:22.703097    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-lg4gt"
Sep 15 11:32:22.999783 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:22.999755    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-lg4gt"]
Sep 15 11:32:23.002525 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:32:23.002500    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbba73fe3_5ba7_4c50_bb0e_7e5efe7b2c5a.slice/crio-3d9d6fbfcaa7afc436697df45a6989d492481d3f21aff236eab6204459c61a96 WatchSource:0}: Error finding container 3d9d6fbfcaa7afc436697df45a6989d492481d3f21aff236eab6204459c61a96: Status 404 returned error can't find the container with id 3d9d6fbfcaa7afc436697df45a6989d492481d3f21aff236eab6204459c61a96
Sep 15 11:32:23.008479 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:23.008449    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:23.257988 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:23.257903    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:23.931458 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:23.931424    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-lg4gt" event={"ID":"bba73fe3-5ba7-4c50-bb0e-7e5efe7b2c5a","Type":"ContainerStarted","Data":"3d9d6fbfcaa7afc436697df45a6989d492481d3f21aff236eab6204459c61a96"}
Sep 15 11:32:24.939202 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:24.939152    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj" event={"ID":"c401d308-5a6e-45fa-be66-f532e52bc884","Type":"ContainerStarted","Data":"869313d7df8c21474da9357b4f21dc8cfe851ab4e141cdc7f669b94ebd105fd1"}
Sep 15 11:32:24.939681 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:24.939282    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:24.966796 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:24.966720    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj" podStartSLOduration=1.934527524 podStartE2EDuration="5.966702151s" totalImagesPullingTime="4.032174627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:32:24.965214 +0000 UTC m=+446.457991148" watchObservedRunningTime="2026-09-15 11:32:24.966702151 +0000 UTC m=+446.459479298"
Sep 15 11:32:25.944967 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:25.944926    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-lg4gt" event={"ID":"bba73fe3-5ba7-4c50-bb0e-7e5efe7b2c5a","Type":"ContainerStarted","Data":"a6023ef2690aabbfeda697eaa4fbdbc1da02387a1b62d0ebd1e3ca897b511deb"}
Sep 15 11:32:25.966937 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:25.966885    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-lg4gt" podStartSLOduration=1.861420117 podStartE2EDuration="3.966870229s" totalImagesPullingTime="2.105450112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:32:25.964984561 +0000 UTC m=+447.457761711" watchObservedRunningTime="2026-09-15 11:32:25.966870229 +0000 UTC m=+447.459647377"
Sep 15 11:32:26.020686 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:26.020649    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:26.738055 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:26.738023    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:26.949540 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:26.949510    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-lg4gt"
Sep 15 11:32:27.127780 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:27.127642    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:27.468634 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:27.468536    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:31.400825 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:31.400786    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:31.500982 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:31.500952    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:31.604833 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:31.604801    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:31.667061 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:31.666974    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:31.805272 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:31.805236    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:31.868222 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:31.868194    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:31.922133 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:31.922066    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-lf6rx"
Sep 15 11:32:35.947345 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:35.947322    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:37.956216 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:37.956187    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-lg4gt"
Sep 15 11:32:38.102868 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.102831    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"]
Sep 15 11:32:38.103036 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.103014    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj" podUID="c401d308-5a6e-45fa-be66-f532e52bc884" containerName="manager" containerID="cri-o://869313d7df8c21474da9357b4f21dc8cfe851ab4e141cdc7f669b94ebd105fd1" gracePeriod=2
Sep 15 11:32:38.114072 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.114044    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"]
Sep 15 11:32:38.123735 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:32:38.123694    2560 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c401d308-5a6e-45fa-be66-f532e52bc884\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-kh9kj\": pods \"kuadrant-operator-controller-manager-76f8f686bf-kh9kj\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:38.133933 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.133906    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"]
Sep 15 11:32:38.134313 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.134294    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c401d308-5a6e-45fa-be66-f532e52bc884" containerName="manager"
Sep 15 11:32:38.134366 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.134318    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c401d308-5a6e-45fa-be66-f532e52bc884" containerName="manager"
Sep 15 11:32:38.134586 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.134432    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c401d308-5a6e-45fa-be66-f532e52bc884" containerName="manager"
Sep 15 11:32:38.137372 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.137354    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"
Sep 15 11:32:38.139229 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:32:38.139197    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-kh9kj\" is forbidden: User \"system:node:ip-10-0-138-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-153.ec2.internal' and this object" podUID="c401d308-5a6e-45fa-be66-f532e52bc884" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:38.156932 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.156908    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"]
Sep 15 11:32:38.207304 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.207194    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9242c30-12f8-4826-b1a5-333212ebf052-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gd9sq\" (UID: \"b9242c30-12f8-4826-b1a5-333212ebf052\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"
Sep 15 11:32:38.207304 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.207242    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4rtj\" (UniqueName: \"kubernetes.io/projected/b9242c30-12f8-4826-b1a5-333212ebf052-kube-api-access-n4rtj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gd9sq\" (UID: \"b9242c30-12f8-4826-b1a5-333212ebf052\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"
Sep 15 11:32:38.308121 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.308086    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9242c30-12f8-4826-b1a5-333212ebf052-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gd9sq\" (UID: \"b9242c30-12f8-4826-b1a5-333212ebf052\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"
Sep 15 11:32:38.308121 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.308125    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4rtj\" (UniqueName: \"kubernetes.io/projected/b9242c30-12f8-4826-b1a5-333212ebf052-kube-api-access-n4rtj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gd9sq\" (UID: \"b9242c30-12f8-4826-b1a5-333212ebf052\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"
Sep 15 11:32:38.308551 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.308527    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9242c30-12f8-4826-b1a5-333212ebf052-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gd9sq\" (UID: \"b9242c30-12f8-4826-b1a5-333212ebf052\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"
Sep 15 11:32:38.314402 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.314375    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"]
Sep 15 11:32:38.317692 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.317669    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"
Sep 15 11:32:38.319709 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:32:38.319682    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-kh9kj\" is forbidden: User \"system:node:ip-10-0-138-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-153.ec2.internal' and this object" podUID="c401d308-5a6e-45fa-be66-f532e52bc884" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:38.329637 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.329618    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4rtj\" (UniqueName: \"kubernetes.io/projected/b9242c30-12f8-4826-b1a5-333212ebf052-kube-api-access-n4rtj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gd9sq\" (UID: \"b9242c30-12f8-4826-b1a5-333212ebf052\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"
Sep 15 11:32:38.336498 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.336472    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"]
Sep 15 11:32:38.348403 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.348384    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:38.379770 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:32:38.379736    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-kh9kj\" is forbidden: User \"system:node:ip-10-0-138-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-153.ec2.internal' and this object" podUID="c401d308-5a6e-45fa-be66-f532e52bc884" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:38.409227 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.409202    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzw26\" (UniqueName: \"kubernetes.io/projected/540f3743-3391-43a0-a98a-178d5e4cee78-kube-api-access-hzw26\") pod \"kuadrant-operator-controller-manager-5769f99d9f-m4jzr\" (UID: \"540f3743-3391-43a0-a98a-178d5e4cee78\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"
Sep 15 11:32:38.409343 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.409240    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/540f3743-3391-43a0-a98a-178d5e4cee78-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-m4jzr\" (UID: \"540f3743-3391-43a0-a98a-178d5e4cee78\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"
Sep 15 11:32:38.484934 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.484858    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"
Sep 15 11:32:38.509868 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.509839    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c401d308-5a6e-45fa-be66-f532e52bc884-extensions-socket-volume\") pod \"c401d308-5a6e-45fa-be66-f532e52bc884\" (UID: \"c401d308-5a6e-45fa-be66-f532e52bc884\") "
Sep 15 11:32:38.510007 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.509916    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgs8l\" (UniqueName: \"kubernetes.io/projected/c401d308-5a6e-45fa-be66-f532e52bc884-kube-api-access-pgs8l\") pod \"c401d308-5a6e-45fa-be66-f532e52bc884\" (UID: \"c401d308-5a6e-45fa-be66-f532e52bc884\") "
Sep 15 11:32:38.510136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.510115    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzw26\" (UniqueName: \"kubernetes.io/projected/540f3743-3391-43a0-a98a-178d5e4cee78-kube-api-access-hzw26\") pod \"kuadrant-operator-controller-manager-5769f99d9f-m4jzr\" (UID: \"540f3743-3391-43a0-a98a-178d5e4cee78\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"
Sep 15 11:32:38.510263 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.510153    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/540f3743-3391-43a0-a98a-178d5e4cee78-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-m4jzr\" (UID: \"540f3743-3391-43a0-a98a-178d5e4cee78\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"
Sep 15 11:32:38.510387 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.510363    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c401d308-5a6e-45fa-be66-f532e52bc884-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c401d308-5a6e-45fa-be66-f532e52bc884" (UID: "c401d308-5a6e-45fa-be66-f532e52bc884"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:32:38.510552 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.510535    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/540f3743-3391-43a0-a98a-178d5e4cee78-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-m4jzr\" (UID: \"540f3743-3391-43a0-a98a-178d5e4cee78\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"
Sep 15 11:32:38.512665 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.512644    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c401d308-5a6e-45fa-be66-f532e52bc884-kube-api-access-pgs8l" (OuterVolumeSpecName: "kube-api-access-pgs8l") pod "c401d308-5a6e-45fa-be66-f532e52bc884" (UID: "c401d308-5a6e-45fa-be66-f532e52bc884"). InnerVolumeSpecName "kube-api-access-pgs8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:32:38.519258 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.519231    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzw26\" (UniqueName: \"kubernetes.io/projected/540f3743-3391-43a0-a98a-178d5e4cee78-kube-api-access-hzw26\") pod \"kuadrant-operator-controller-manager-5769f99d9f-m4jzr\" (UID: \"540f3743-3391-43a0-a98a-178d5e4cee78\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"
Sep 15 11:32:38.610836 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.610798    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c401d308-5a6e-45fa-be66-f532e52bc884-extensions-socket-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:32:38.610836 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.610839    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgs8l\" (UniqueName: \"kubernetes.io/projected/c401d308-5a6e-45fa-be66-f532e52bc884-kube-api-access-pgs8l\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:32:38.644630 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.644598    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"
Sep 15 11:32:38.769613 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.769585    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"]
Sep 15 11:32:38.772015 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:32:38.771992    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9242c30_12f8_4826_b1a5_333212ebf052.slice/crio-c575be2a816296c62f2a38cfaa5cff0d2a381573a95d6a24dddf45d7c1daa33b WatchSource:0}: Error finding container c575be2a816296c62f2a38cfaa5cff0d2a381573a95d6a24dddf45d7c1daa33b: Status 404 returned error can't find the container with id c575be2a816296c62f2a38cfaa5cff0d2a381573a95d6a24dddf45d7c1daa33b
Sep 15 11:32:38.921076 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:38.921035    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"]
Sep 15 11:32:38.924083 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:32:38.924055    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod540f3743_3391_43a0_a98a_178d5e4cee78.slice/crio-b580f6e7cd61080f3d3f42667e617f2367cddca3259502d9d675c55c5403a2ce WatchSource:0}: Error finding container b580f6e7cd61080f3d3f42667e617f2367cddca3259502d9d675c55c5403a2ce: Status 404 returned error can't find the container with id b580f6e7cd61080f3d3f42667e617f2367cddca3259502d9d675c55c5403a2ce
Sep 15 11:32:39.010250 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.010201    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr" event={"ID":"540f3743-3391-43a0-a98a-178d5e4cee78","Type":"ContainerStarted","Data":"519f7a86b0ec28b7e14b53c584cdfe61c4b8584ce32f7ce5b555063d6d30ec82"}
Sep 15 11:32:39.010712 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.010300    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr" event={"ID":"540f3743-3391-43a0-a98a-178d5e4cee78","Type":"ContainerStarted","Data":"b580f6e7cd61080f3d3f42667e617f2367cddca3259502d9d675c55c5403a2ce"}
Sep 15 11:32:39.010712 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.010330    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"
Sep 15 11:32:39.011892 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.011863    2560 generic.go:356] "Generic (PLEG): container finished" podID="c401d308-5a6e-45fa-be66-f532e52bc884" containerID="869313d7df8c21474da9357b4f21dc8cfe851ab4e141cdc7f669b94ebd105fd1" exitCode=0
Sep 15 11:32:39.012011 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.011917    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:39.012011 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.011983    2560 scope.go:117] "RemoveContainer" containerID="869313d7df8c21474da9357b4f21dc8cfe851ab4e141cdc7f669b94ebd105fd1"
Sep 15 11:32:39.012607 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:32:39.012581    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-kh9kj\" is forbidden: User \"system:node:ip-10-0-138-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-153.ec2.internal' and this object" podUID="c401d308-5a6e-45fa-be66-f532e52bc884" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-kh9kj"
Sep 15 11:32:39.014264 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.014232    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq" event={"ID":"b9242c30-12f8-4826-b1a5-333212ebf052","Type":"ContainerStarted","Data":"f6f9fa586ff823834b02a1f366d230ca202815817add6167ea09faa8207f23f2"}
Sep 15 11:32:39.014350 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.014271    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq" event={"ID":"b9242c30-12f8-4826-b1a5-333212ebf052","Type":"ContainerStarted","Data":"c575be2a816296c62f2a38cfaa5cff0d2a381573a95d6a24dddf45d7c1daa33b"}
Sep 15 11:32:39.014410 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.014391    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"
Sep 15 11:32:39.021463 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.021446    2560 scope.go:117] "RemoveContainer" containerID="869313d7df8c21474da9357b4f21dc8cfe851ab4e141cdc7f669b94ebd105fd1"
Sep 15 11:32:39.021740 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:32:39.021721    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"869313d7df8c21474da9357b4f21dc8cfe851ab4e141cdc7f669b94ebd105fd1\": container with ID starting with 869313d7df8c21474da9357b4f21dc8cfe851ab4e141cdc7f669b94ebd105fd1 not found: ID does not exist" containerID="869313d7df8c21474da9357b4f21dc8cfe851ab4e141cdc7f669b94ebd105fd1"
Sep 15 11:32:39.021801 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.021751    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"869313d7df8c21474da9357b4f21dc8cfe851ab4e141cdc7f669b94ebd105fd1"} err="failed to get container status \"869313d7df8c21474da9357b4f21dc8cfe851ab4e141cdc7f669b94ebd105fd1\": rpc error: code = NotFound desc = could not find container \"869313d7df8c21474da9357b4f21dc8cfe851ab4e141cdc7f669b94ebd105fd1\": container with ID starting with 869313d7df8c21474da9357b4f21dc8cfe851ab4e141cdc7f669b94ebd105fd1 not found: ID does not exist"
Sep 15 11:32:39.033863 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.033833    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c401d308-5a6e-45fa-be66-f532e52bc884" path="/var/lib/kubelet/pods/c401d308-5a6e-45fa-be66-f532e52bc884/volumes"
Sep 15 11:32:39.083614 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.083544    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr" podStartSLOduration=1.0835273029999999 podStartE2EDuration="1.083527303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:32:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:32:39.081228872 +0000 UTC m=+460.574006033" watchObservedRunningTime="2026-09-15 11:32:39.083527303 +0000 UTC m=+460.576304455"
Sep 15 11:32:39.125365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.125315    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq" podStartSLOduration=1.125299958 podStartE2EDuration="1.125299958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:32:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:32:39.123415078 +0000 UTC m=+460.616192228" watchObservedRunningTime="2026-09-15 11:32:39.125299958 +0000 UTC m=+460.618077105"
Sep 15 11:32:39.450888 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.450861    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:39.548297 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:39.548257    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:42.428863 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:42.428822    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:42.531961 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:42.531926    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:45.053984 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:45.053913    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:45.158647 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:45.158612    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:50.022327 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:50.022293    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"
Sep 15 11:32:50.022714 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:50.022364    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"
Sep 15 11:32:50.092249 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:50.092220    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"]
Sep 15 11:32:50.092440 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:50.092418    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq" podUID="b9242c30-12f8-4826-b1a5-333212ebf052" containerName="manager" containerID="cri-o://f6f9fa586ff823834b02a1f366d230ca202815817add6167ea09faa8207f23f2" gracePeriod=10
Sep 15 11:32:50.325188 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:50.325164    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"
Sep 15 11:32:50.388128 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:50.388102    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9242c30-12f8-4826-b1a5-333212ebf052-extensions-socket-volume\") pod \"b9242c30-12f8-4826-b1a5-333212ebf052\" (UID: \"b9242c30-12f8-4826-b1a5-333212ebf052\") "
Sep 15 11:32:50.388316 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:50.388144    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4rtj\" (UniqueName: \"kubernetes.io/projected/b9242c30-12f8-4826-b1a5-333212ebf052-kube-api-access-n4rtj\") pod \"b9242c30-12f8-4826-b1a5-333212ebf052\" (UID: \"b9242c30-12f8-4826-b1a5-333212ebf052\") "
Sep 15 11:32:50.388608 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:50.388554    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9242c30-12f8-4826-b1a5-333212ebf052-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b9242c30-12f8-4826-b1a5-333212ebf052" (UID: "b9242c30-12f8-4826-b1a5-333212ebf052"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:32:50.390288 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:50.390257    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9242c30-12f8-4826-b1a5-333212ebf052-kube-api-access-n4rtj" (OuterVolumeSpecName: "kube-api-access-n4rtj") pod "b9242c30-12f8-4826-b1a5-333212ebf052" (UID: "b9242c30-12f8-4826-b1a5-333212ebf052"). InnerVolumeSpecName "kube-api-access-n4rtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:32:50.455593 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:50.455551    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:50.488727 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:50.488701    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9242c30-12f8-4826-b1a5-333212ebf052-extensions-socket-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:32:50.488727 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:50.488725    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4rtj\" (UniqueName: \"kubernetes.io/projected/b9242c30-12f8-4826-b1a5-333212ebf052-kube-api-access-n4rtj\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:32:50.612736 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:50.612698    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:50.843000 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:50.842960    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:50.945355 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:50.945277    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:51.074454 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:51.074421    2560 generic.go:356] "Generic (PLEG): container finished" podID="b9242c30-12f8-4826-b1a5-333212ebf052" containerID="f6f9fa586ff823834b02a1f366d230ca202815817add6167ea09faa8207f23f2" exitCode=0
Sep 15 11:32:51.074847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:51.074462    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"
Sep 15 11:32:51.074847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:51.074482    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq" event={"ID":"b9242c30-12f8-4826-b1a5-333212ebf052","Type":"ContainerDied","Data":"f6f9fa586ff823834b02a1f366d230ca202815817add6167ea09faa8207f23f2"}
Sep 15 11:32:51.074847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:51.074534    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq" event={"ID":"b9242c30-12f8-4826-b1a5-333212ebf052","Type":"ContainerDied","Data":"c575be2a816296c62f2a38cfaa5cff0d2a381573a95d6a24dddf45d7c1daa33b"}
Sep 15 11:32:51.074847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:51.074583    2560 scope.go:117] "RemoveContainer" containerID="f6f9fa586ff823834b02a1f366d230ca202815817add6167ea09faa8207f23f2"
Sep 15 11:32:51.082737 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:51.082703    2560 scope.go:117] "RemoveContainer" containerID="f6f9fa586ff823834b02a1f366d230ca202815817add6167ea09faa8207f23f2"
Sep 15 11:32:51.082984 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:32:51.082964    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6f9fa586ff823834b02a1f366d230ca202815817add6167ea09faa8207f23f2\": container with ID starting with f6f9fa586ff823834b02a1f366d230ca202815817add6167ea09faa8207f23f2 not found: ID does not exist" containerID="f6f9fa586ff823834b02a1f366d230ca202815817add6167ea09faa8207f23f2"
Sep 15 11:32:51.083052 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:51.082991    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6f9fa586ff823834b02a1f366d230ca202815817add6167ea09faa8207f23f2"} err="failed to get container status \"f6f9fa586ff823834b02a1f366d230ca202815817add6167ea09faa8207f23f2\": rpc error: code = NotFound desc = could not find container \"f6f9fa586ff823834b02a1f366d230ca202815817add6167ea09faa8207f23f2\": container with ID starting with f6f9fa586ff823834b02a1f366d230ca202815817add6167ea09faa8207f23f2 not found: ID does not exist"
Sep 15 11:32:51.110321 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:51.110296    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"]
Sep 15 11:32:51.115088 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:51.115053    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gd9sq"]
Sep 15 11:32:51.271945 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:51.271870    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:51.812151 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:51.812113    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:51.941306 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:51.941274    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:53.034141 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:53.034107    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9242c30-12f8-4826-b1a5-333212ebf052" path="/var/lib/kubelet/pods/b9242c30-12f8-4826-b1a5-333212ebf052/volumes"
Sep 15 11:32:57.684746 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:57.684717    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:57.743148 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:57.743116    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:59.242725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:59.242637    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:32:59.338089 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:32:59.338059    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:01.533627 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:01.533585    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:02.234550 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:02.234506    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:02.336928 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:02.336896    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:06.464072 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.464040    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"]
Sep 15 11:33:06.464521 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.464385    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9242c30-12f8-4826-b1a5-333212ebf052" containerName="manager"
Sep 15 11:33:06.464521 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.464397    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9242c30-12f8-4826-b1a5-333212ebf052" containerName="manager"
Sep 15 11:33:06.464521 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.464442    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9242c30-12f8-4826-b1a5-333212ebf052" containerName="manager"
Sep 15 11:33:06.467999 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.467977    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.470654 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.470630    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-rdvhj\""
Sep 15 11:33:06.489100 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.489075    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"]
Sep 15 11:33:06.513421 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.513396    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:06.522451 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.522422    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.522607 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.522464    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.522607 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.522499    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.522607 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.522552    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.522607 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.522585    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zdbx\" (UniqueName: \"kubernetes.io/projected/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-kube-api-access-8zdbx\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.522607 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.522604    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.522842 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.522623    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.522842 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.522718    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.522842 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.522763    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.623904 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.623864    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.623904 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.623906    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zdbx\" (UniqueName: \"kubernetes.io/projected/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-kube-api-access-8zdbx\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.624116 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.623929    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.624116 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.624048    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.624230 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.624184    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.624296 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.624246    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.624296 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.624247    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.624407 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.624297    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.624407 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.624322    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.624407 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.624343    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.624407 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.624368    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.624407 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.624378    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.624680 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.624625    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.624957 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.624940    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.626509 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.626482    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.626756 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.626739    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.633783 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.633759    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.633783 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.633775    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zdbx\" (UniqueName: \"kubernetes.io/projected/f4e91b9c-b31e-48a0-bb33-c4cd19625eea-kube-api-access-8zdbx\") pod \"maas-default-gateway-openshift-default-5ccb968c66-pgr4n\" (UID: \"f4e91b9c-b31e-48a0-bb33-c4cd19625eea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:06.778323 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:06.778240    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:07.056431 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:07.056327    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"]
Sep 15 11:33:07.060789 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:33:07.060758    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4e91b9c_b31e_48a0_bb33_c4cd19625eea.slice/crio-7332389ad6adddb33605ab9277197a8603440e6a2c4352aff051df34421780fc WatchSource:0}: Error finding container 7332389ad6adddb33605ab9277197a8603440e6a2c4352aff051df34421780fc: Status 404 returned error can't find the container with id 7332389ad6adddb33605ab9277197a8603440e6a2c4352aff051df34421780fc
Sep 15 11:33:07.152399 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:07.152365    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n" event={"ID":"f4e91b9c-b31e-48a0-bb33-c4cd19625eea","Type":"ContainerStarted","Data":"7332389ad6adddb33605ab9277197a8603440e6a2c4352aff051df34421780fc"}
Sep 15 11:33:07.152717 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:07.152687    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:09.725343 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:09.725308    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:33:09.725647 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:09.725378    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:33:09.725647 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:09.725407    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:33:10.168004 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:10.167970    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n" event={"ID":"f4e91b9c-b31e-48a0-bb33-c4cd19625eea","Type":"ContainerStarted","Data":"97deff89388a73925dce545e695816e9206ddee9bfb482a4568c651476130616"}
Sep 15 11:33:10.190233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:10.190180    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n" podStartSLOduration=1.5280321780000001 podStartE2EDuration="4.190164886s" totalImagesPullingTime="2.662132708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:33:10.188072451 +0000 UTC m=+491.680849611" watchObservedRunningTime="2026-09-15 11:33:10.190164886 +0000 UTC m=+491.682942033"
Sep 15 11:33:10.779398 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:10.779365    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:10.784090 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:10.784065    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:11.172722 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.172688    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:11.173892 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.173870    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:33:11.225895 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.225854    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hblq4"]
Sep 15 11:33:11.229325 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.229307    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hblq4"
Sep 15 11:33:11.231453 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.231430    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bgv4b\""
Sep 15 11:33:11.231645 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.231618    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 15 11:33:11.239096 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.239069    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hblq4"]
Sep 15 11:33:11.239833 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.239210    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:11.325384 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.325353    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hblq4"]
Sep 15 11:33:11.328277 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.328246    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:11.365569 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.365533    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgdml\" (UniqueName: \"kubernetes.io/projected/486033d4-d4cc-4106-9ccd-196693c3ba15-kube-api-access-bgdml\") pod \"limitador-limitador-8588749977-hblq4\" (UID: \"486033d4-d4cc-4106-9ccd-196693c3ba15\") " pod="kuadrant-system/limitador-limitador-8588749977-hblq4"
Sep 15 11:33:11.365712 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.365605    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/486033d4-d4cc-4106-9ccd-196693c3ba15-config-file\") pod \"limitador-limitador-8588749977-hblq4\" (UID: \"486033d4-d4cc-4106-9ccd-196693c3ba15\") " pod="kuadrant-system/limitador-limitador-8588749977-hblq4"
Sep 15 11:33:11.406552 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.406520    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:11.467027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.466943    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgdml\" (UniqueName: \"kubernetes.io/projected/486033d4-d4cc-4106-9ccd-196693c3ba15-kube-api-access-bgdml\") pod \"limitador-limitador-8588749977-hblq4\" (UID: \"486033d4-d4cc-4106-9ccd-196693c3ba15\") " pod="kuadrant-system/limitador-limitador-8588749977-hblq4"
Sep 15 11:33:11.467027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.466990    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/486033d4-d4cc-4106-9ccd-196693c3ba15-config-file\") pod \"limitador-limitador-8588749977-hblq4\" (UID: \"486033d4-d4cc-4106-9ccd-196693c3ba15\") " pod="kuadrant-system/limitador-limitador-8588749977-hblq4"
Sep 15 11:33:11.467609 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.467590    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/486033d4-d4cc-4106-9ccd-196693c3ba15-config-file\") pod \"limitador-limitador-8588749977-hblq4\" (UID: \"486033d4-d4cc-4106-9ccd-196693c3ba15\") " pod="kuadrant-system/limitador-limitador-8588749977-hblq4"
Sep 15 11:33:11.495056 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.495030    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgdml\" (UniqueName: \"kubernetes.io/projected/486033d4-d4cc-4106-9ccd-196693c3ba15-kube-api-access-bgdml\") pod \"limitador-limitador-8588749977-hblq4\" (UID: \"486033d4-d4cc-4106-9ccd-196693c3ba15\") " pod="kuadrant-system/limitador-limitador-8588749977-hblq4"
Sep 15 11:33:11.541885 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.541850    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hblq4"
Sep 15 11:33:11.813122 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:11.813097    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hblq4"]
Sep 15 11:33:11.815812 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:33:11.815783    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod486033d4_d4cc_4106_9ccd_196693c3ba15.slice/crio-0949ec96b6652891491362c568bedef52acc77914f841a3efcf64c2003d21a82 WatchSource:0}: Error finding container 0949ec96b6652891491362c568bedef52acc77914f841a3efcf64c2003d21a82: Status 404 returned error can't find the container with id 0949ec96b6652891491362c568bedef52acc77914f841a3efcf64c2003d21a82
Sep 15 11:33:12.178950 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:12.178911    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hblq4" event={"ID":"486033d4-d4cc-4106-9ccd-196693c3ba15","Type":"ContainerStarted","Data":"0949ec96b6652891491362c568bedef52acc77914f841a3efcf64c2003d21a82"}
Sep 15 11:33:12.334239 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:12.334204    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:12.457789 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:12.457698    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:20.215737 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:20.215695    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hblq4" event={"ID":"486033d4-d4cc-4106-9ccd-196693c3ba15","Type":"ContainerStarted","Data":"e4d03f6663adf6ba375dda794fc6bad35f17a6aadc885162af1835bda55f1082"}
Sep 15 11:33:20.216122 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:20.215774    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-hblq4"
Sep 15 11:33:20.233437 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:20.233384    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-hblq4" podStartSLOduration=1.863558187 podStartE2EDuration="9.233368597s" totalImagesPullingTime="7.36981041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:33:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:33:20.231887256 +0000 UTC m=+501.724664418" watchObservedRunningTime="2026-09-15 11:33:20.233368597 +0000 UTC m=+501.726145772"
Sep 15 11:33:22.669713 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:22.669674    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:22.745204 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:22.745175    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:24.966424 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:24.966393    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hblq4"]
Sep 15 11:33:24.966854 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:24.966598    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-hblq4" podUID="486033d4-d4cc-4106-9ccd-196693c3ba15" containerName="limitador" containerID="cri-o://e4d03f6663adf6ba375dda794fc6bad35f17a6aadc885162af1835bda55f1082" gracePeriod=30
Sep 15 11:33:24.968669 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:24.968647    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-hblq4"
Sep 15 11:33:25.503117 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:25.503094    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hblq4"
Sep 15 11:33:25.583240 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:25.583157    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgdml\" (UniqueName: \"kubernetes.io/projected/486033d4-d4cc-4106-9ccd-196693c3ba15-kube-api-access-bgdml\") pod \"486033d4-d4cc-4106-9ccd-196693c3ba15\" (UID: \"486033d4-d4cc-4106-9ccd-196693c3ba15\") "
Sep 15 11:33:25.583240 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:25.583225    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/486033d4-d4cc-4106-9ccd-196693c3ba15-config-file\") pod \"486033d4-d4cc-4106-9ccd-196693c3ba15\" (UID: \"486033d4-d4cc-4106-9ccd-196693c3ba15\") "
Sep 15 11:33:25.583578 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:25.583536    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/486033d4-d4cc-4106-9ccd-196693c3ba15-config-file" (OuterVolumeSpecName: "config-file") pod "486033d4-d4cc-4106-9ccd-196693c3ba15" (UID: "486033d4-d4cc-4106-9ccd-196693c3ba15"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:33:25.585301 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:25.585277    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/486033d4-d4cc-4106-9ccd-196693c3ba15-kube-api-access-bgdml" (OuterVolumeSpecName: "kube-api-access-bgdml") pod "486033d4-d4cc-4106-9ccd-196693c3ba15" (UID: "486033d4-d4cc-4106-9ccd-196693c3ba15"). InnerVolumeSpecName "kube-api-access-bgdml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:33:25.684505 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:25.684472    2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/486033d4-d4cc-4106-9ccd-196693c3ba15-config-file\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:33:25.684505 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:25.684502    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgdml\" (UniqueName: \"kubernetes.io/projected/486033d4-d4cc-4106-9ccd-196693c3ba15-kube-api-access-bgdml\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:33:26.243355 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:26.243324    2560 generic.go:356] "Generic (PLEG): container finished" podID="486033d4-d4cc-4106-9ccd-196693c3ba15" containerID="e4d03f6663adf6ba375dda794fc6bad35f17a6aadc885162af1835bda55f1082" exitCode=0
Sep 15 11:33:26.243793 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:26.243368    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hblq4"
Sep 15 11:33:26.243793 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:26.243413    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hblq4" event={"ID":"486033d4-d4cc-4106-9ccd-196693c3ba15","Type":"ContainerDied","Data":"e4d03f6663adf6ba375dda794fc6bad35f17a6aadc885162af1835bda55f1082"}
Sep 15 11:33:26.243793 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:26.243460    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hblq4" event={"ID":"486033d4-d4cc-4106-9ccd-196693c3ba15","Type":"ContainerDied","Data":"0949ec96b6652891491362c568bedef52acc77914f841a3efcf64c2003d21a82"}
Sep 15 11:33:26.243793 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:26.243481    2560 scope.go:117] "RemoveContainer" containerID="e4d03f6663adf6ba375dda794fc6bad35f17a6aadc885162af1835bda55f1082"
Sep 15 11:33:26.252603 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:26.252586    2560 scope.go:117] "RemoveContainer" containerID="e4d03f6663adf6ba375dda794fc6bad35f17a6aadc885162af1835bda55f1082"
Sep 15 11:33:26.252869 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:33:26.252851    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4d03f6663adf6ba375dda794fc6bad35f17a6aadc885162af1835bda55f1082\": container with ID starting with e4d03f6663adf6ba375dda794fc6bad35f17a6aadc885162af1835bda55f1082 not found: ID does not exist" containerID="e4d03f6663adf6ba375dda794fc6bad35f17a6aadc885162af1835bda55f1082"
Sep 15 11:33:26.252917 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:26.252880    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4d03f6663adf6ba375dda794fc6bad35f17a6aadc885162af1835bda55f1082"} err="failed to get container status \"e4d03f6663adf6ba375dda794fc6bad35f17a6aadc885162af1835bda55f1082\": rpc error: code = NotFound desc = could not find container \"e4d03f6663adf6ba375dda794fc6bad35f17a6aadc885162af1835bda55f1082\": container with ID starting with e4d03f6663adf6ba375dda794fc6bad35f17a6aadc885162af1835bda55f1082 not found: ID does not exist"
Sep 15 11:33:26.266378 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:26.266358    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hblq4"]
Sep 15 11:33:26.279475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:26.279449    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hblq4"]
Sep 15 11:33:27.034365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:27.034326    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="486033d4-d4cc-4106-9ccd-196693c3ba15" path="/var/lib/kubelet/pods/486033d4-d4cc-4106-9ccd-196693c3ba15/volumes"
Sep 15 11:33:27.395738 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:27.395696    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:27.490750 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:27.490708    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:27.823454 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:27.823369    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5st7g"]
Sep 15 11:33:27.823982 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:27.823960    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="486033d4-d4cc-4106-9ccd-196693c3ba15" containerName="limitador"
Sep 15 11:33:27.824057 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:27.823986    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="486033d4-d4cc-4106-9ccd-196693c3ba15" containerName="limitador"
Sep 15 11:33:27.824119 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:27.824076    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="486033d4-d4cc-4106-9ccd-196693c3ba15" containerName="limitador"
Sep 15 11:33:27.828663 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:27.828637    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5st7g"
Sep 15 11:33:27.830936 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:27.830911    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 15 11:33:27.831071 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:27.830955    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-jllzd\""
Sep 15 11:33:27.831071 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:27.830977    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 11:33:27.831513 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:27.831496    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 11:33:27.835888 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:27.835866    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5st7g"]
Sep 15 11:33:27.888583 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:27.888530    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:28.004923 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:28.004891    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfzph\" (UniqueName: \"kubernetes.io/projected/e0334b89-a740-419c-83b2-e55f5bcd9855-kube-api-access-zfzph\") pod \"postgres-69fb4594cf-5st7g\" (UID: \"e0334b89-a740-419c-83b2-e55f5bcd9855\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5st7g"
Sep 15 11:33:28.004923 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:28.004925    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e0334b89-a740-419c-83b2-e55f5bcd9855-data\") pod \"postgres-69fb4594cf-5st7g\" (UID: \"e0334b89-a740-419c-83b2-e55f5bcd9855\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5st7g"
Sep 15 11:33:28.106281 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:28.106245    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e0334b89-a740-419c-83b2-e55f5bcd9855-data\") pod \"postgres-69fb4594cf-5st7g\" (UID: \"e0334b89-a740-419c-83b2-e55f5bcd9855\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5st7g"
Sep 15 11:33:28.106438 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:28.106358    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfzph\" (UniqueName: \"kubernetes.io/projected/e0334b89-a740-419c-83b2-e55f5bcd9855-kube-api-access-zfzph\") pod \"postgres-69fb4594cf-5st7g\" (UID: \"e0334b89-a740-419c-83b2-e55f5bcd9855\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5st7g"
Sep 15 11:33:28.106651 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:28.106631    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e0334b89-a740-419c-83b2-e55f5bcd9855-data\") pod \"postgres-69fb4594cf-5st7g\" (UID: \"e0334b89-a740-419c-83b2-e55f5bcd9855\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5st7g"
Sep 15 11:33:28.114405 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:28.114377    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfzph\" (UniqueName: \"kubernetes.io/projected/e0334b89-a740-419c-83b2-e55f5bcd9855-kube-api-access-zfzph\") pod \"postgres-69fb4594cf-5st7g\" (UID: \"e0334b89-a740-419c-83b2-e55f5bcd9855\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5st7g"
Sep 15 11:33:28.142349 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:28.142326    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5st7g"
Sep 15 11:33:28.416501 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:28.416474    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5st7g"]
Sep 15 11:33:28.418700 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:33:28.418666    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0334b89_a740_419c_83b2_e55f5bcd9855.slice/crio-1e436129910bc57c751caddf303c8a60691336afe37856711b07d10f6ea4af15 WatchSource:0}: Error finding container 1e436129910bc57c751caddf303c8a60691336afe37856711b07d10f6ea4af15: Status 404 returned error can't find the container with id 1e436129910bc57c751caddf303c8a60691336afe37856711b07d10f6ea4af15
Sep 15 11:33:29.265426 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:29.265388    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5st7g" event={"ID":"e0334b89-a740-419c-83b2-e55f5bcd9855","Type":"ContainerStarted","Data":"1e436129910bc57c751caddf303c8a60691336afe37856711b07d10f6ea4af15"}
Sep 15 11:33:33.287052 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:33.286957    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5st7g" event={"ID":"e0334b89-a740-419c-83b2-e55f5bcd9855","Type":"ContainerStarted","Data":"1072e115ff94136980da88d8f992edcc6bae7cdfe7c23418f801326d048be2a0"}
Sep 15 11:33:33.287052 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:33.287028    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5st7g"
Sep 15 11:33:33.305044 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:33.304998    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5st7g" podStartSLOduration=1.7566148099999999 podStartE2EDuration="6.304983241s" totalImagesPullingTime="4.548368431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:33:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:33:33.303546879 +0000 UTC m=+514.796324039" watchObservedRunningTime="2026-09-15 11:33:33.304983241 +0000 UTC m=+514.797760389"
Sep 15 11:33:39.320259 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:39.320226    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5st7g"
Sep 15 11:33:39.641274 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:39.641242    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:39.972286 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:39.972196    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:46.805511 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.805468    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:46.843062 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.843037    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b8d54ddd7-mlvhf"]
Sep 15 11:33:46.851186 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.851162    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:33:46.853622 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.853598    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 15 11:33:46.853732 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.853659    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 15 11:33:46.853864 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.853849    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-969bz\""
Sep 15 11:33:46.853935 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.853920    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 15 11:33:46.864732 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.864707    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-webhook-cert\") pod \"maas-controller-6b8d54ddd7-mlvhf\" (UID: \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\") " pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:33:46.864862 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.864770    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tkbs\" (UniqueName: \"kubernetes.io/projected/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-kube-api-access-5tkbs\") pod \"maas-controller-6b8d54ddd7-mlvhf\" (UID: \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\") " pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:33:46.864862 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.864832    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-metrics-certs\") pod \"maas-controller-6b8d54ddd7-mlvhf\" (UID: \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\") " pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:33:46.865991 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.865968    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b8d54ddd7-mlvhf"]
Sep 15 11:33:46.965638 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.965608    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tkbs\" (UniqueName: \"kubernetes.io/projected/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-kube-api-access-5tkbs\") pod \"maas-controller-6b8d54ddd7-mlvhf\" (UID: \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\") " pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:33:46.965800 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.965644    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-metrics-certs\") pod \"maas-controller-6b8d54ddd7-mlvhf\" (UID: \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\") " pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:33:46.965800 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.965697    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-webhook-cert\") pod \"maas-controller-6b8d54ddd7-mlvhf\" (UID: \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\") " pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:33:46.965883 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:33:46.965817    2560 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 15 11:33:46.965924 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:33:46.965884    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-metrics-certs podName:265c3247-1f6d-4bb0-8aa7-7f44e818c00a nodeName:}" failed. No retries permitted until 2026-09-15 11:33:47.465870329 +0000 UTC m=+528.958647460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-metrics-certs") pod "maas-controller-6b8d54ddd7-mlvhf" (UID: "265c3247-1f6d-4bb0-8aa7-7f44e818c00a") : secret "maas-controller-metrics-tls" not found
Sep 15 11:33:46.968083 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.968063    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-webhook-cert\") pod \"maas-controller-6b8d54ddd7-mlvhf\" (UID: \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\") " pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:33:46.976171 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:46.976147    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tkbs\" (UniqueName: \"kubernetes.io/projected/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-kube-api-access-5tkbs\") pod \"maas-controller-6b8d54ddd7-mlvhf\" (UID: \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\") " pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:33:47.036114 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.036078    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:47.044392 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.044368    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c8fb6dfc5-q6nnf"]
Sep 15 11:33:47.048146 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.048131    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 11:33:47.061309 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.061255    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c8fb6dfc5-q6nnf"]
Sep 15 11:33:47.066592 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.066542    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a050db2-5725-4e16-97b0-1da533d92eee-webhook-cert\") pod \"maas-controller-5c8fb6dfc5-q6nnf\" (UID: \"9a050db2-5725-4e16-97b0-1da533d92eee\") " pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 11:33:47.066695 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.066611    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mbr7\" (UniqueName: \"kubernetes.io/projected/9a050db2-5725-4e16-97b0-1da533d92eee-kube-api-access-8mbr7\") pod \"maas-controller-5c8fb6dfc5-q6nnf\" (UID: \"9a050db2-5725-4e16-97b0-1da533d92eee\") " pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 11:33:47.066695 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.066636    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a050db2-5725-4e16-97b0-1da533d92eee-metrics-certs\") pod \"maas-controller-5c8fb6dfc5-q6nnf\" (UID: \"9a050db2-5725-4e16-97b0-1da533d92eee\") " pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 11:33:47.113999 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.113970    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:47.167112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.167085    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mbr7\" (UniqueName: \"kubernetes.io/projected/9a050db2-5725-4e16-97b0-1da533d92eee-kube-api-access-8mbr7\") pod \"maas-controller-5c8fb6dfc5-q6nnf\" (UID: \"9a050db2-5725-4e16-97b0-1da533d92eee\") " pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 11:33:47.167217 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.167118    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a050db2-5725-4e16-97b0-1da533d92eee-metrics-certs\") pod \"maas-controller-5c8fb6dfc5-q6nnf\" (UID: \"9a050db2-5725-4e16-97b0-1da533d92eee\") " pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 11:33:47.167217 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.167185    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a050db2-5725-4e16-97b0-1da533d92eee-webhook-cert\") pod \"maas-controller-5c8fb6dfc5-q6nnf\" (UID: \"9a050db2-5725-4e16-97b0-1da533d92eee\") " pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 11:33:47.167311 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:33:47.167300    2560 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 15 11:33:47.167376 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:33:47.167365    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a050db2-5725-4e16-97b0-1da533d92eee-metrics-certs podName:9a050db2-5725-4e16-97b0-1da533d92eee nodeName:}" failed. No retries permitted until 2026-09-15 11:33:47.667346408 +0000 UTC m=+529.160123549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9a050db2-5725-4e16-97b0-1da533d92eee-metrics-certs") pod "maas-controller-5c8fb6dfc5-q6nnf" (UID: "9a050db2-5725-4e16-97b0-1da533d92eee") : secret "maas-controller-metrics-tls" not found
Sep 15 11:33:47.169508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.169480    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a050db2-5725-4e16-97b0-1da533d92eee-webhook-cert\") pod \"maas-controller-5c8fb6dfc5-q6nnf\" (UID: \"9a050db2-5725-4e16-97b0-1da533d92eee\") " pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 11:33:47.177785 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.177760    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mbr7\" (UniqueName: \"kubernetes.io/projected/9a050db2-5725-4e16-97b0-1da533d92eee-kube-api-access-8mbr7\") pod \"maas-controller-5c8fb6dfc5-q6nnf\" (UID: \"9a050db2-5725-4e16-97b0-1da533d92eee\") " pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 11:33:47.228502 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.228479    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:47.468613 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.468546    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-metrics-certs\") pod \"maas-controller-6b8d54ddd7-mlvhf\" (UID: \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\") " pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:33:47.471001 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.470976    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-metrics-certs\") pod \"maas-controller-6b8d54ddd7-mlvhf\" (UID: \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\") " pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:33:47.671118 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.671085    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a050db2-5725-4e16-97b0-1da533d92eee-metrics-certs\") pod \"maas-controller-5c8fb6dfc5-q6nnf\" (UID: \"9a050db2-5725-4e16-97b0-1da533d92eee\") " pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 11:33:47.673356 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.673337    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a050db2-5725-4e16-97b0-1da533d92eee-metrics-certs\") pod \"maas-controller-5c8fb6dfc5-q6nnf\" (UID: \"9a050db2-5725-4e16-97b0-1da533d92eee\") " pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 11:33:47.761089 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.761020    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:33:47.957945 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:47.957912    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 11:33:48.038421 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:48.038395    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b8d54ddd7-mlvhf"]
Sep 15 11:33:48.040721 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:33:48.040695    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod265c3247_1f6d_4bb0_8aa7_7f44e818c00a.slice/crio-fca83ca1a57dddaeb15a93f619c3e9ed9e3226d5b732e63df0a8dee98bbb5a38 WatchSource:0}: Error finding container fca83ca1a57dddaeb15a93f619c3e9ed9e3226d5b732e63df0a8dee98bbb5a38: Status 404 returned error can't find the container with id fca83ca1a57dddaeb15a93f619c3e9ed9e3226d5b732e63df0a8dee98bbb5a38
Sep 15 11:33:48.232252 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:48.232227    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c8fb6dfc5-q6nnf"]
Sep 15 11:33:48.234444 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:33:48.234413    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a050db2_5725_4e16_97b0_1da533d92eee.slice/crio-0f3bfce466aa13eac248ba74dea4af8c1c38fc6a544216865cb93a4ec0b7c858 WatchSource:0}: Error finding container 0f3bfce466aa13eac248ba74dea4af8c1c38fc6a544216865cb93a4ec0b7c858: Status 404 returned error can't find the container with id 0f3bfce466aa13eac248ba74dea4af8c1c38fc6a544216865cb93a4ec0b7c858
Sep 15 11:33:48.365119 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:48.365088    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf" event={"ID":"9a050db2-5725-4e16-97b0-1da533d92eee","Type":"ContainerStarted","Data":"0f3bfce466aa13eac248ba74dea4af8c1c38fc6a544216865cb93a4ec0b7c858"}
Sep 15 11:33:48.366259 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:48.366238    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf" event={"ID":"265c3247-1f6d-4bb0-8aa7-7f44e818c00a","Type":"ContainerStarted","Data":"fca83ca1a57dddaeb15a93f619c3e9ed9e3226d5b732e63df0a8dee98bbb5a38"}
Sep 15 11:33:50.729228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:50.729188    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:50.814526 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:50.814431    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:33:51.383767 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:51.383682    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf" event={"ID":"9a050db2-5725-4e16-97b0-1da533d92eee","Type":"ContainerStarted","Data":"9dfd33dbdec7fb2da3d8071eb83df1ab7cd300ba266a8c0360b5b951f6f5aa64"}
Sep 15 11:33:51.383927 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:51.383814    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 11:33:51.385247 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:51.385219    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf" event={"ID":"265c3247-1f6d-4bb0-8aa7-7f44e818c00a","Type":"ContainerStarted","Data":"c06c33e73f80bec5c8073b07d7d1a6fc4c8bcdeb255774b992835043643f4cfe"}
Sep 15 11:33:51.385361 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:51.385298    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:33:51.406657 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:51.406605    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf" podStartSLOduration=2.098426431 podStartE2EDuration="4.406585921s" totalImagesPullingTime="2.30815949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:33:51.404398435 +0000 UTC m=+532.897175584" watchObservedRunningTime="2026-09-15 11:33:51.406585921 +0000 UTC m=+532.899363078"
Sep 15 11:33:51.424380 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:51.424333    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf" podStartSLOduration=2.93003743 podStartE2EDuration="5.424316062s" totalImagesPullingTime="2.494278632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:33:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:33:51.423030788 +0000 UTC m=+532.915807938" watchObservedRunningTime="2026-09-15 11:33:51.424316062 +0000 UTC m=+532.917093210"
Sep 15 11:33:51.737517 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:33:51.737421    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:34:00.700041 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:00.700001    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:34:00.797915 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:00.797883    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:34:02.392704 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:02.392670    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 11:34:02.393185 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:02.392938    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:34:02.461056 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:02.461024    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b8d54ddd7-mlvhf"]
Sep 15 11:34:02.461303 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:02.461272    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf" podUID="265c3247-1f6d-4bb0-8aa7-7f44e818c00a" containerName="manager" containerID="cri-o://c06c33e73f80bec5c8073b07d7d1a6fc4c8bcdeb255774b992835043643f4cfe" gracePeriod=30
Sep 15 11:34:02.706438 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:02.706412    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:34:02.803419 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:02.803395    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-webhook-cert\") pod \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\" (UID: \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\") "
Sep 15 11:34:02.803573 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:02.803438    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-metrics-certs\") pod \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\" (UID: \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\") "
Sep 15 11:34:02.803573 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:02.803479    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tkbs\" (UniqueName: \"kubernetes.io/projected/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-kube-api-access-5tkbs\") pod \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\" (UID: \"265c3247-1f6d-4bb0-8aa7-7f44e818c00a\") "
Sep 15 11:34:02.805549 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:02.805517    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "265c3247-1f6d-4bb0-8aa7-7f44e818c00a" (UID: "265c3247-1f6d-4bb0-8aa7-7f44e818c00a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:34:02.805681 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:02.805584    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-kube-api-access-5tkbs" (OuterVolumeSpecName: "kube-api-access-5tkbs") pod "265c3247-1f6d-4bb0-8aa7-7f44e818c00a" (UID: "265c3247-1f6d-4bb0-8aa7-7f44e818c00a"). InnerVolumeSpecName "kube-api-access-5tkbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:34:02.805681 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:02.805594    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "265c3247-1f6d-4bb0-8aa7-7f44e818c00a" (UID: "265c3247-1f6d-4bb0-8aa7-7f44e818c00a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:34:02.904190 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:02.904163    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-metrics-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:34:02.904190 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:02.904186    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tkbs\" (UniqueName: \"kubernetes.io/projected/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-kube-api-access-5tkbs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:34:02.904190 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:02.904195    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/265c3247-1f6d-4bb0-8aa7-7f44e818c00a-webhook-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:34:03.445190 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:03.445153    2560 generic.go:356] "Generic (PLEG): container finished" podID="265c3247-1f6d-4bb0-8aa7-7f44e818c00a" containerID="c06c33e73f80bec5c8073b07d7d1a6fc4c8bcdeb255774b992835043643f4cfe" exitCode=0
Sep 15 11:34:03.445685 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:03.445201    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf"
Sep 15 11:34:03.445685 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:03.445244    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf" event={"ID":"265c3247-1f6d-4bb0-8aa7-7f44e818c00a","Type":"ContainerDied","Data":"c06c33e73f80bec5c8073b07d7d1a6fc4c8bcdeb255774b992835043643f4cfe"}
Sep 15 11:34:03.445685 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:03.445299    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b8d54ddd7-mlvhf" event={"ID":"265c3247-1f6d-4bb0-8aa7-7f44e818c00a","Type":"ContainerDied","Data":"fca83ca1a57dddaeb15a93f619c3e9ed9e3226d5b732e63df0a8dee98bbb5a38"}
Sep 15 11:34:03.445685 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:03.445322    2560 scope.go:117] "RemoveContainer" containerID="c06c33e73f80bec5c8073b07d7d1a6fc4c8bcdeb255774b992835043643f4cfe"
Sep 15 11:34:03.455677 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:03.455660    2560 scope.go:117] "RemoveContainer" containerID="c06c33e73f80bec5c8073b07d7d1a6fc4c8bcdeb255774b992835043643f4cfe"
Sep 15 11:34:03.455941 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:34:03.455921    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c06c33e73f80bec5c8073b07d7d1a6fc4c8bcdeb255774b992835043643f4cfe\": container with ID starting with c06c33e73f80bec5c8073b07d7d1a6fc4c8bcdeb255774b992835043643f4cfe not found: ID does not exist" containerID="c06c33e73f80bec5c8073b07d7d1a6fc4c8bcdeb255774b992835043643f4cfe"
Sep 15 11:34:03.456011 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:03.455953    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c06c33e73f80bec5c8073b07d7d1a6fc4c8bcdeb255774b992835043643f4cfe"} err="failed to get container status \"c06c33e73f80bec5c8073b07d7d1a6fc4c8bcdeb255774b992835043643f4cfe\": rpc error: code = NotFound desc = could not find container \"c06c33e73f80bec5c8073b07d7d1a6fc4c8bcdeb255774b992835043643f4cfe\": container with ID starting with c06c33e73f80bec5c8073b07d7d1a6fc4c8bcdeb255774b992835043643f4cfe not found: ID does not exist"
Sep 15 11:34:03.468051 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:03.468025    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b8d54ddd7-mlvhf"]
Sep 15 11:34:03.469502 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:03.469482    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b8d54ddd7-mlvhf"]
Sep 15 11:34:05.034475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:05.034440    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="265c3247-1f6d-4bb0-8aa7-7f44e818c00a" path="/var/lib/kubelet/pods/265c3247-1f6d-4bb0-8aa7-7f44e818c00a/volumes"
Sep 15 11:34:19.885834 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:19.885797    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:34:19.986446 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:19.986411    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:34:21.369502 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.369463    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:34:21.471414 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.471372    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:34:21.609191 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.609157    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"]
Sep 15 11:34:21.609501 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.609489    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="265c3247-1f6d-4bb0-8aa7-7f44e818c00a" containerName="manager"
Sep 15 11:34:21.609550 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.609502    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="265c3247-1f6d-4bb0-8aa7-7f44e818c00a" containerName="manager"
Sep 15 11:34:21.609616 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.609591    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="265c3247-1f6d-4bb0-8aa7-7f44e818c00a" containerName="manager"
Sep 15 11:34:21.628495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.628420    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"]
Sep 15 11:34:21.628651 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.628524    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:21.631758 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.631738    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 15 11:34:21.632410 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.632392    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bjt69\""
Sep 15 11:34:21.632532 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.632420    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 15 11:34:21.762242 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.762206    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df018f63-5cc1-4614-9fae-ca23d05bd862-metrics-certs\") pod \"maas-api-57d8d6c56b-t9gmq\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") " pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:21.762426 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.762258    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df018f63-5cc1-4614-9fae-ca23d05bd862-maas-api-tls\") pod \"maas-api-57d8d6c56b-t9gmq\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") " pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:21.762426 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.762311    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqkzj\" (UniqueName: \"kubernetes.io/projected/df018f63-5cc1-4614-9fae-ca23d05bd862-kube-api-access-rqkzj\") pod \"maas-api-57d8d6c56b-t9gmq\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") " pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:21.762426 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.762381    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df018f63-5cc1-4614-9fae-ca23d05bd862-openshift-service-ca\") pod \"maas-api-57d8d6c56b-t9gmq\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") " pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:21.763226 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.763203    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:34:21.862935 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.862903    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df018f63-5cc1-4614-9fae-ca23d05bd862-metrics-certs\") pod \"maas-api-57d8d6c56b-t9gmq\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") " pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:21.863101 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.862953    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df018f63-5cc1-4614-9fae-ca23d05bd862-maas-api-tls\") pod \"maas-api-57d8d6c56b-t9gmq\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") " pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:21.863101 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.862977    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqkzj\" (UniqueName: \"kubernetes.io/projected/df018f63-5cc1-4614-9fae-ca23d05bd862-kube-api-access-rqkzj\") pod \"maas-api-57d8d6c56b-t9gmq\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") " pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:21.863101 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.863007    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df018f63-5cc1-4614-9fae-ca23d05bd862-openshift-service-ca\") pod \"maas-api-57d8d6c56b-t9gmq\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") " pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:21.863764 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.863738    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df018f63-5cc1-4614-9fae-ca23d05bd862-openshift-service-ca\") pod \"maas-api-57d8d6c56b-t9gmq\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") " pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:21.865489 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.865459    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df018f63-5cc1-4614-9fae-ca23d05bd862-maas-api-tls\") pod \"maas-api-57d8d6c56b-t9gmq\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") " pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:21.865644 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.865625    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df018f63-5cc1-4614-9fae-ca23d05bd862-metrics-certs\") pod \"maas-api-57d8d6c56b-t9gmq\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") " pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:21.873264 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.873238    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqkzj\" (UniqueName: \"kubernetes.io/projected/df018f63-5cc1-4614-9fae-ca23d05bd862-kube-api-access-rqkzj\") pod \"maas-api-57d8d6c56b-t9gmq\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") " pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:21.892029 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.891955    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:34:21.939143 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.939112    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:21.992171 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:21.991425    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:34:22.092617 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:22.092581    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9"]
Sep 15 11:34:22.096227 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:22.096210    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9"
Sep 15 11:34:22.098848 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:22.098826    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-46fpx\""
Sep 15 11:34:22.099179 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:22.099160    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 15 11:34:22.107934 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:22.107903    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9"]
Sep 15 11:34:22.221772 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:22.221746    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"]
Sep 15 11:34:22.223910 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:34:22.223882    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf018f63_5cc1_4614_9fae_ca23d05bd862.slice/crio-aa92729cd3e394b54f5ac2e0d4ea297dfa3608f54f615a366bf73b4881dfc211 WatchSource:0}: Error finding container aa92729cd3e394b54f5ac2e0d4ea297dfa3608f54f615a366bf73b4881dfc211: Status 404 returned error can't find the container with id aa92729cd3e394b54f5ac2e0d4ea297dfa3608f54f615a366bf73b4881dfc211
Sep 15 11:34:22.266430 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:22.266392    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bgqm\" (UniqueName: \"kubernetes.io/projected/abe51807-1597-4321-8154-f4e185610305-kube-api-access-5bgqm\") pod \"payload-pre-processing-59fbc47bf8-f9vl9\" (UID: \"abe51807-1597-4321-8154-f4e185610305\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9"
Sep 15 11:34:22.266602 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:22.266436    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abe51807-1597-4321-8154-f4e185610305-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-f9vl9\" (UID: \"abe51807-1597-4321-8154-f4e185610305\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9"
Sep 15 11:34:22.367140 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:22.367092    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bgqm\" (UniqueName: \"kubernetes.io/projected/abe51807-1597-4321-8154-f4e185610305-kube-api-access-5bgqm\") pod \"payload-pre-processing-59fbc47bf8-f9vl9\" (UID: \"abe51807-1597-4321-8154-f4e185610305\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9"
Sep 15 11:34:22.367361 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:22.367158    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abe51807-1597-4321-8154-f4e185610305-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-f9vl9\" (UID: \"abe51807-1597-4321-8154-f4e185610305\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9"
Sep 15 11:34:22.367925 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:22.367905    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abe51807-1597-4321-8154-f4e185610305-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-f9vl9\" (UID: \"abe51807-1597-4321-8154-f4e185610305\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9"
Sep 15 11:34:22.374773 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:22.374745    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bgqm\" (UniqueName: \"kubernetes.io/projected/abe51807-1597-4321-8154-f4e185610305-kube-api-access-5bgqm\") pod \"payload-pre-processing-59fbc47bf8-f9vl9\" (UID: \"abe51807-1597-4321-8154-f4e185610305\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9"
Sep 15 11:34:22.407701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:22.407664    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9"
Sep 15 11:34:22.539916 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:22.539824    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq" event={"ID":"df018f63-5cc1-4614-9fae-ca23d05bd862","Type":"ContainerStarted","Data":"aa92729cd3e394b54f5ac2e0d4ea297dfa3608f54f615a366bf73b4881dfc211"}
Sep 15 11:34:22.680897 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:22.680748    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9"]
Sep 15 11:34:22.683033 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:34:22.683006    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabe51807_1597_4321_8154_f4e185610305.slice/crio-250f0f204529c8815e70ce8c54533dca339aacd4367ab81e9d30fce1ff8bd752 WatchSource:0}: Error finding container 250f0f204529c8815e70ce8c54533dca339aacd4367ab81e9d30fce1ff8bd752: Status 404 returned error can't find the container with id 250f0f204529c8815e70ce8c54533dca339aacd4367ab81e9d30fce1ff8bd752
Sep 15 11:34:23.549026 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:23.548990    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9" event={"ID":"abe51807-1597-4321-8154-f4e185610305","Type":"ContainerStarted","Data":"250f0f204529c8815e70ce8c54533dca339aacd4367ab81e9d30fce1ff8bd752"}
Sep 15 11:34:26.568315 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:26.568277    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9" event={"ID":"abe51807-1597-4321-8154-f4e185610305","Type":"ContainerStarted","Data":"cebb57ecba6d72129fe8f071509907b5c4402050443aad7d26596049f33c97c6"}
Sep 15 11:34:26.568840 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:26.568366    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9"
Sep 15 11:34:26.569862 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:26.569838    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq" event={"ID":"df018f63-5cc1-4614-9fae-ca23d05bd862","Type":"ContainerStarted","Data":"12c6532a2b7ac2495a76f87225015649484705c886a6f165ba1fb267d4a0643a"}
Sep 15 11:34:26.569986 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:26.569972    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:26.584490 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:26.584449    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9" podStartSLOduration=1.505865146 podStartE2EDuration="4.584434699s" totalImagesPullingTime="3.078569553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:34:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:34:26.583597192 +0000 UTC m=+568.076374336" watchObservedRunningTime="2026-09-15 11:34:26.584434699 +0000 UTC m=+568.077211860"
Sep 15 11:34:26.600476 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:26.600430    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq" podStartSLOduration=2.10910255 podStartE2EDuration="5.600398695s" totalImagesPullingTime="3.491296145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:34:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:34:26.599240335 +0000 UTC m=+568.092017483" watchObservedRunningTime="2026-09-15 11:34:26.600398695 +0000 UTC m=+568.093175845"
Sep 15 11:34:31.596102 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:31.596073    2560 generic.go:356] "Generic (PLEG): container finished" podID="f4e91b9c-b31e-48a0-bb33-c4cd19625eea" containerID="97deff89388a73925dce545e695816e9206ddee9bfb482a4568c651476130616" exitCode=0
Sep 15 11:34:31.596513 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:31.596150    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n" event={"ID":"f4e91b9c-b31e-48a0-bb33-c4cd19625eea","Type":"ContainerDied","Data":"97deff89388a73925dce545e695816e9206ddee9bfb482a4568c651476130616"}
Sep 15 11:34:31.596582 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:31.596550    2560 scope.go:117] "RemoveContainer" containerID="97deff89388a73925dce545e695816e9206ddee9bfb482a4568c651476130616"
Sep 15 11:34:31.597577 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:31.597533    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:34:31.597643 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:31.597616    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:34:31.597682 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:31.597650    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:34:32.581584 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:32.581535    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 11:34:32.610071 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:32.610028    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n" event={"ID":"f4e91b9c-b31e-48a0-bb33-c4cd19625eea","Type":"ContainerStarted","Data":"69111af9fe3cf01720853f49c4cf5c211693f6af11e63616b2aa0a64be400a46"}
Sep 15 11:34:32.779358 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:32.779312    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:34:32.779555 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:32.779530    2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-pgr4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 15 11:34:32.779644 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:32.779614    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n" podUID="f4e91b9c-b31e-48a0-bb33-c4cd19625eea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 15 11:34:33.594401 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:33.594367    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:34:33.779228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:33.779195    2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-pgr4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 15 11:34:33.779631 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:33.779247    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n" podUID="f4e91b9c-b31e-48a0-bb33-c4cd19625eea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 15 11:34:34.100506 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:34.100467    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:34:34.779502 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:34.779466    2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-pgr4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 15 11:34:34.779899 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:34.779529    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n" podUID="f4e91b9c-b31e-48a0-bb33-c4cd19625eea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 15 11:34:35.779426 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:35.779390    2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-pgr4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 15 11:34:35.779644 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:35.779454    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n" podUID="f4e91b9c-b31e-48a0-bb33-c4cd19625eea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 15 11:34:36.477656 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:36.477623    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:34:36.779535 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:36.779445    2560 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-pgr4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 15 11:34:36.779535 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:36.779506    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n" podUID="f4e91b9c-b31e-48a0-bb33-c4cd19625eea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 15 11:34:37.578302 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:37.578264    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f9vl9"
Sep 15 11:34:37.782302 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:37.782268    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:34:37.782754 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:37.782521    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:34:37.783179 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:37.783163    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-pgr4n"
Sep 15 11:34:47.041797 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:47.041742    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:34:58.963690 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:58.963656    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:34:58.967471 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:34:58.967448    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:35:09.348879 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.348845    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"]
Sep 15 11:35:09.355910 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.355888    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.358059 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.358032    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 15 11:35:09.358689 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.358670    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 15 11:35:09.358796 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.358730    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ghj57\""
Sep 15 11:35:09.358796 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.358734    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 15 11:35:09.364546 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.364524    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"]
Sep 15 11:35:09.387327 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.387296    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:35:09.487688 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.487654    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d29b9b99-a2d6-4d10-acab-e488bc14b721-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.487893 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.487714    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d29b9b99-a2d6-4d10-acab-e488bc14b721-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.487893 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.487811    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d29b9b99-a2d6-4d10-acab-e488bc14b721-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.487893 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.487849    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sjqs\" (UniqueName: \"kubernetes.io/projected/d29b9b99-a2d6-4d10-acab-e488bc14b721-kube-api-access-6sjqs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.487893 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.487880    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d29b9b99-a2d6-4d10-acab-e488bc14b721-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.488129 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.487940    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d29b9b99-a2d6-4d10-acab-e488bc14b721-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.588992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.588954    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d29b9b99-a2d6-4d10-acab-e488bc14b721-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.589189 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.589028    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d29b9b99-a2d6-4d10-acab-e488bc14b721-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.589189 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.589053    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sjqs\" (UniqueName: \"kubernetes.io/projected/d29b9b99-a2d6-4d10-acab-e488bc14b721-kube-api-access-6sjqs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.589189 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.589081    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d29b9b99-a2d6-4d10-acab-e488bc14b721-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.589189 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.589135    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d29b9b99-a2d6-4d10-acab-e488bc14b721-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.589389 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.589200    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d29b9b99-a2d6-4d10-acab-e488bc14b721-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.589455 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.589431    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d29b9b99-a2d6-4d10-acab-e488bc14b721-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.589522 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.589474    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d29b9b99-a2d6-4d10-acab-e488bc14b721-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.589613 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.589543    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d29b9b99-a2d6-4d10-acab-e488bc14b721-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.592057 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.592031    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d29b9b99-a2d6-4d10-acab-e488bc14b721-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.592217 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.592199    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d29b9b99-a2d6-4d10-acab-e488bc14b721-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.597917 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.597898    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sjqs\" (UniqueName: \"kubernetes.io/projected/d29b9b99-a2d6-4d10-acab-e488bc14b721-kube-api-access-6sjqs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mcxtm\" (UID: \"d29b9b99-a2d6-4d10-acab-e488bc14b721\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.666495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.666419    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:09.942025 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:09.941870    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"]
Sep 15 11:35:09.944666 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:35:09.944636    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd29b9b99_a2d6_4d10_acab_e488bc14b721.slice/crio-77395baf9bc6793d15b9039ed890af11a93864ee65f48da75bb60f2dc28a2ef1 WatchSource:0}: Error finding container 77395baf9bc6793d15b9039ed890af11a93864ee65f48da75bb60f2dc28a2ef1: Status 404 returned error can't find the container with id 77395baf9bc6793d15b9039ed890af11a93864ee65f48da75bb60f2dc28a2ef1
Sep 15 11:35:10.800839 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:10.800796    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm" event={"ID":"d29b9b99-a2d6-4d10-acab-e488bc14b721","Type":"ContainerStarted","Data":"77395baf9bc6793d15b9039ed890af11a93864ee65f48da75bb60f2dc28a2ef1"}
Sep 15 11:35:11.808103 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:11.808060    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm" event={"ID":"d29b9b99-a2d6-4d10-acab-e488bc14b721","Type":"ContainerStarted","Data":"6a9a7e9969db5c364b545bbce80ee72418916714b248d966bbe67b4f06303284"}
Sep 15 11:35:11.826115 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:11.826066    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm" podStartSLOduration=1.6830411 podStartE2EDuration="2.826050878s" totalImagesPullingTime="1.143009778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:35:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:35:11.824244369 +0000 UTC m=+613.317021517" watchObservedRunningTime="2026-09-15 11:35:11.826050878 +0000 UTC m=+613.318828026"
Sep 15 11:35:19.667464 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:19.667415    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:19.667975 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:19.667493    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:19.682224 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:19.682192    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:19.856262 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:19.856228    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mcxtm"
Sep 15 11:35:23.389500 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:23.389399    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:35:40.397114 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:40.397077    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:35:46.770378 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.770341    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"]
Sep 15 11:35:46.776775 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.776752    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.784773 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.784749    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 15 11:35:46.797143 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.797114    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"]
Sep 15 11:35:46.813159 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.813124    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c23523bc-df65-48a2-9e95-60caab38b465-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.813272 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.813221    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4nkt\" (UniqueName: \"kubernetes.io/projected/c23523bc-df65-48a2-9e95-60caab38b465-kube-api-access-b4nkt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.813272 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.813262    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c23523bc-df65-48a2-9e95-60caab38b465-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.813345 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.813295    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c23523bc-df65-48a2-9e95-60caab38b465-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.813383 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.813366    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c23523bc-df65-48a2-9e95-60caab38b465-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.813453 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.813432    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c23523bc-df65-48a2-9e95-60caab38b465-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.878015 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.877978    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:35:46.914268 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.914239    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c23523bc-df65-48a2-9e95-60caab38b465-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.914446 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.914291    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c23523bc-df65-48a2-9e95-60caab38b465-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.914446 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.914328    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4nkt\" (UniqueName: \"kubernetes.io/projected/c23523bc-df65-48a2-9e95-60caab38b465-kube-api-access-b4nkt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.914446 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.914352    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c23523bc-df65-48a2-9e95-60caab38b465-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.914446 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.914374    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c23523bc-df65-48a2-9e95-60caab38b465-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.914446 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.914415    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c23523bc-df65-48a2-9e95-60caab38b465-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.914810 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.914784    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c23523bc-df65-48a2-9e95-60caab38b465-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.914876 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.914818    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c23523bc-df65-48a2-9e95-60caab38b465-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.914926 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.914897    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c23523bc-df65-48a2-9e95-60caab38b465-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.916772 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.916745    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c23523bc-df65-48a2-9e95-60caab38b465-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.916957 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.916939    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c23523bc-df65-48a2-9e95-60caab38b465-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:46.922260 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:46.922239    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4nkt\" (UniqueName: \"kubernetes.io/projected/c23523bc-df65-48a2-9e95-60caab38b465-kube-api-access-b4nkt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5\" (UID: \"c23523bc-df65-48a2-9e95-60caab38b465\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:47.086616 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:47.086524    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:47.363310 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:47.363197    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"]
Sep 15 11:35:47.366070 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:35:47.366025    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23523bc_df65_48a2_9e95_60caab38b465.slice/crio-cec155d592c0fb387c13a461dcf47aeb3d9e1a4508979eb48ea916e7b146c7d7 WatchSource:0}: Error finding container cec155d592c0fb387c13a461dcf47aeb3d9e1a4508979eb48ea916e7b146c7d7: Status 404 returned error can't find the container with id cec155d592c0fb387c13a461dcf47aeb3d9e1a4508979eb48ea916e7b146c7d7
Sep 15 11:35:47.367429 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:47.367413    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 11:35:47.982111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:47.982061    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5" event={"ID":"c23523bc-df65-48a2-9e95-60caab38b465","Type":"ContainerStarted","Data":"4c3aa125f6beabdc43249c1caebc8637737730bc791e7d2e1d41d465746d3fee"}
Sep 15 11:35:47.982498 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:47.982162    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5" event={"ID":"c23523bc-df65-48a2-9e95-60caab38b465","Type":"ContainerStarted","Data":"cec155d592c0fb387c13a461dcf47aeb3d9e1a4508979eb48ea916e7b146c7d7"}
Sep 15 11:35:47.999691 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:47.999635    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5" podStartSLOduration=1.821533902 podStartE2EDuration="1.99961956s" totalImagesPullingTime="178.085658ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:35:47.998830294 +0000 UTC m=+649.491607446" watchObservedRunningTime="2026-09-15 11:35:47.99961956 +0000 UTC m=+649.492396744"
Sep 15 11:35:57.087170 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:57.087138    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:57.087669 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:57.087200    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:57.099593 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:57.099541    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:35:58.042863 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:35:58.042831    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5"
Sep 15 11:36:02.828784 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:36:02.828747    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:36:12.913962 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:36:12.913921    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:36:50.267554 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:36:50.267508    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:36:50.327855 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:36:50.327823    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:37:43.812000 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:37:43.811950    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:37:43.930112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:37:43.930074    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:37:44.993918 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:37:44.993877    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:37:45.092530 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:37:45.092496    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:37:49.320394 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:37:49.320360    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:02.937701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:02.937663    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:03.039187 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.039148    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:03.143973 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.143942    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:03.340673 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.340632    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:03.387895 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.387866    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"]
Sep 15 11:38:03.391556 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.391534    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.394116 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.394093    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-b8fbdd-openshift-default-dockercfg-rwtbj\""
Sep 15 11:38:03.409705 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.409682    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"]
Sep 15 11:38:03.458400 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.458365    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:03.490156 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.490119    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-workload-socket\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.490342 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.490198    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-credential-socket\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.490342 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.490231    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-data\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.490342 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.490305    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0582ae44-03a9-4a95-841d-ef551a58fc19-istiod-ca-cert\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.490342 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.490340    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-token\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.490676 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.490386    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4gjx\" (UniqueName: \"kubernetes.io/projected/0582ae44-03a9-4a95-841d-ef551a58fc19-kube-api-access-r4gjx\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.490676 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.490536    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-podinfo\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.490676 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.490608    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-envoy\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.490676 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.490665    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-workload-certs\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.496592 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.496554    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"]
Sep 15 11:38:03.500216 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.500193    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.503276 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.503254    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-6e996c-openshift-default-dockercfg-qwv4v\""
Sep 15 11:38:03.517667 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.517641    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"]
Sep 15 11:38:03.551169 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.551144    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:03.591182 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591101    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-podinfo\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.591182 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591138    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-envoy\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.591386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591185    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-workload-certs\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.591386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591216    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-workload-socket\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.591386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591249    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czs7d\" (UniqueName: \"kubernetes.io/projected/8f737c66-5417-491d-8c79-9a4649e2eb7a-kube-api-access-czs7d\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.591386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591279    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-credential-socket\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.591386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591300    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-data\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.591386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591327    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-workload-socket\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.591386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591361    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-podinfo\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.591867 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591406    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0582ae44-03a9-4a95-841d-ef551a58fc19-istiod-ca-cert\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.591867 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591433    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-envoy\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.591867 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591712    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-token\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.591867 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591793    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-token\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.591867 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591821    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-credential-socket\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.591867 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591845    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-workload-certs\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.592174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591883    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-data\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.592174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591924    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4gjx\" (UniqueName: \"kubernetes.io/projected/0582ae44-03a9-4a95-841d-ef551a58fc19-kube-api-access-r4gjx\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.592174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.591962    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f737c66-5417-491d-8c79-9a4649e2eb7a-istiod-ca-cert\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.592406 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.592386    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-credential-socket\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.592789 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.592765    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-data\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.593059 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.593025    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-workload-socket\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.593153 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.593127    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-workload-certs\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.593212 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.593174    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0582ae44-03a9-4a95-841d-ef551a58fc19-istiod-ca-cert\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.594643 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.594611    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-envoy\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.595021 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.595000    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-podinfo\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.605796 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.605753    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-token\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.607682 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.607660    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4gjx\" (UniqueName: \"kubernetes.io/projected/0582ae44-03a9-4a95-841d-ef551a58fc19-kube-api-access-r4gjx\") pod \"e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") " pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.693226 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.693185    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czs7d\" (UniqueName: \"kubernetes.io/projected/8f737c66-5417-491d-8c79-9a4649e2eb7a-kube-api-access-czs7d\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.693226 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.693227    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-workload-socket\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.693456 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.693258    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-podinfo\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.693456 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.693298    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-envoy\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.693456 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.693320    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-token\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.693456 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.693356    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-credential-socket\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.693456 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.693382    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-workload-certs\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.693456 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.693414    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-data\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.693843 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.693461    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f737c66-5417-491d-8c79-9a4649e2eb7a-istiod-ca-cert\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.693843 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.693657    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-workload-socket\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.693843 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.693754    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-credential-socket\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.693843 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.693834    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-workload-certs\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.694020 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.693998    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-data\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.694247 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.694226    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f737c66-5417-491d-8c79-9a4649e2eb7a-istiod-ca-cert\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.695777 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.695753    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-envoy\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.696011 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.695990    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-podinfo\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.701898 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.701863    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-token\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.702643 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.702618    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:03.708072 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.708053    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czs7d\" (UniqueName: \"kubernetes.io/projected/8f737c66-5417-491d-8c79-9a4649e2eb7a-kube-api-access-czs7d\") pod \"e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") " pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.729708 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.729678    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:03.811513 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.811484    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:03.842698 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.842598    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:03.943645 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.943613    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:03.982283 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.982260    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"]
Sep 15 11:38:03.984333 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:38:03.984304    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0582ae44_03a9_4a95_841d_ef551a58fc19.slice/crio-d546bcee4e666f6940a6a5e4ce0f155c5e4299624d0f7f2d7e99b0fcd8f5d542 WatchSource:0}: Error finding container d546bcee4e666f6940a6a5e4ce0f155c5e4299624d0f7f2d7e99b0fcd8f5d542: Status 404 returned error can't find the container with id d546bcee4e666f6940a6a5e4ce0f155c5e4299624d0f7f2d7e99b0fcd8f5d542
Sep 15 11:38:03.986345 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.986316    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:38:03.986414 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.986376    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:38:03.986414 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:03.986407    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:38:04.090311 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:04.090271    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"]
Sep 15 11:38:04.095741 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:38:04.095712    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f737c66_5417_491d_8c79_9a4649e2eb7a.slice/crio-d5a23d981f2f6f33c9c735bb33de3c6bd9cbc4f6197b330d09c062c442e779f3 WatchSource:0}: Error finding container d5a23d981f2f6f33c9c735bb33de3c6bd9cbc4f6197b330d09c062c442e779f3: Status 404 returned error can't find the container with id d5a23d981f2f6f33c9c735bb33de3c6bd9cbc4f6197b330d09c062c442e779f3
Sep 15 11:38:04.098417 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:04.098381    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:38:04.098520 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:04.098468    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:38:04.098520 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:04.098508    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:38:04.340035 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:04.339997    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:04.441882 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:04.441783    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:04.648743 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:04.648704    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz" event={"ID":"8f737c66-5417-491d-8c79-9a4649e2eb7a","Type":"ContainerStarted","Data":"023166f619ab846f2d9d82fbc15449a5a5fd741cb302ca251c60e1de04c7b612"}
Sep 15 11:38:04.650108 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:04.649520    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz" event={"ID":"8f737c66-5417-491d-8c79-9a4649e2eb7a","Type":"ContainerStarted","Data":"d5a23d981f2f6f33c9c735bb33de3c6bd9cbc4f6197b330d09c062c442e779f3"}
Sep 15 11:38:04.651243 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:04.651211    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4" event={"ID":"0582ae44-03a9-4a95-841d-ef551a58fc19","Type":"ContainerStarted","Data":"ae364f9002995116d5d754ab15b26327b1225fe850b5c6ac87ee9cff0207d797"}
Sep 15 11:38:04.651337 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:04.651254    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4" event={"ID":"0582ae44-03a9-4a95-841d-ef551a58fc19","Type":"ContainerStarted","Data":"d546bcee4e666f6940a6a5e4ce0f155c5e4299624d0f7f2d7e99b0fcd8f5d542"}
Sep 15 11:38:04.675692 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:04.675652    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz" podStartSLOduration=1.6756396709999999 podStartE2EDuration="1.675639671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:38:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:38:04.672662194 +0000 UTC m=+786.165439345" watchObservedRunningTime="2026-09-15 11:38:04.675639671 +0000 UTC m=+786.168416819"
Sep 15 11:38:04.692419 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:04.692297    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4" podStartSLOduration=1.6922794410000002 podStartE2EDuration="1.692279441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:38:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:38:04.691953382 +0000 UTC m=+786.184730530" watchObservedRunningTime="2026-09-15 11:38:04.692279441 +0000 UTC m=+786.185056610"
Sep 15 11:38:04.702803 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:04.702780    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:04.811921 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:04.811881    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:04.838169 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:04.838133    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:05.709060 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:05.709026    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:05.817832 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:05.817794    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:06.365791 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:06.365755    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:06.474795 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:06.474756    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:06.575312 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:06.575282    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:06.661942 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:06.661844    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:06.661942 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:06.661904    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:06.662660 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:06.662635    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:06.662776 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:06.662705    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:06.794616 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:06.794574    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:07.022318 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:07.022223    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:07.129013 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:07.128977    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:07.237019 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:07.236985    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:08.896231 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:08.896189    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"]
Sep 15 11:38:08.900959 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:08.900936    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:08.916249 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:08.916226    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-9fc4b9-serving-cert\""
Sep 15 11:38:08.946863 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:08.946834    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"]
Sep 15 11:38:08.947182 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:08.947149    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd2cf\" (UniqueName: \"kubernetes.io/projected/cb53f9c3-1536-4739-aed4-49bc9814d1ec-kube-api-access-fd2cf\") pod \"maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:08.947311 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:08.947292    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb53f9c3-1536-4739-aed4-49bc9814d1ec-metrics-certs\") pod \"maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:08.947375 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:08.947346    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb53f9c3-1536-4739-aed4-49bc9814d1ec-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:08.947434 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:08.947377    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb53f9c3-1536-4739-aed4-49bc9814d1ec-maas-api-tls\") pod \"maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:08.976871 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:08.976827    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:09.048230 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.048198    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd2cf\" (UniqueName: \"kubernetes.io/projected/cb53f9c3-1536-4739-aed4-49bc9814d1ec-kube-api-access-fd2cf\") pod \"maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:09.048417 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.048304    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb53f9c3-1536-4739-aed4-49bc9814d1ec-metrics-certs\") pod \"maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:09.048417 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.048341    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb53f9c3-1536-4739-aed4-49bc9814d1ec-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:09.048417 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.048368    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb53f9c3-1536-4739-aed4-49bc9814d1ec-maas-api-tls\") pod \"maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:09.048636 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:38:09.048487    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-serving-cert: secret "maas-api-e2e-inf-a-9fc4b9-serving-cert" not found
Sep 15 11:38:09.048636 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:38:09.048594    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb53f9c3-1536-4739-aed4-49bc9814d1ec-maas-api-tls podName:cb53f9c3-1536-4739-aed4-49bc9814d1ec nodeName:}" failed. No retries permitted until 2026-09-15 11:38:09.548573566 +0000 UTC m=+791.041350708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cb53f9c3-1536-4739-aed4-49bc9814d1ec-maas-api-tls") pod "maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz" (UID: "cb53f9c3-1536-4739-aed4-49bc9814d1ec") : secret "maas-api-e2e-inf-a-9fc4b9-serving-cert" not found
Sep 15 11:38:09.049131 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.049108    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb53f9c3-1536-4739-aed4-49bc9814d1ec-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:09.050948 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.050924    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb53f9c3-1536-4739-aed4-49bc9814d1ec-metrics-certs\") pod \"maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:09.066956 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.066930    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd2cf\" (UniqueName: \"kubernetes.io/projected/cb53f9c3-1536-4739-aed4-49bc9814d1ec-kube-api-access-fd2cf\") pod \"maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:09.097785 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.097752    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:09.310367 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.310283    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:09.553920 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.553886    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb53f9c3-1536-4739-aed4-49bc9814d1ec-maas-api-tls\") pod \"maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:09.556344 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.556315    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb53f9c3-1536-4739-aed4-49bc9814d1ec-maas-api-tls\") pod \"maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:09.580306 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.580247    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:09.668322 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.668289    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:09.690525 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.690493    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"]
Sep 15 11:38:09.694366 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.694345    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"
Sep 15 11:38:09.696502 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.696482    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-9fc4b9\""
Sep 15 11:38:09.696888 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.696873    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-9fc4b9-dockercfg-zjd4v\""
Sep 15 11:38:09.706226 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.706205    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"]
Sep 15 11:38:09.728838 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.728810    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"]
Sep 15 11:38:09.732299 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.732280    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"
Sep 15 11:38:09.743190 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.743168    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"]
Sep 15 11:38:09.755856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.755835    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6\" (UID: \"7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"
Sep 15 11:38:09.755954 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.755911    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3deab65e-55c8-4ca9-94a5-0e31678f10f9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb\" (UID: \"3deab65e-55c8-4ca9-94a5-0e31678f10f9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"
Sep 15 11:38:09.755954 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.755933    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fc54\" (UniqueName: \"kubernetes.io/projected/3deab65e-55c8-4ca9-94a5-0e31678f10f9-kube-api-access-2fc54\") pod \"payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb\" (UID: \"3deab65e-55c8-4ca9-94a5-0e31678f10f9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"
Sep 15 11:38:09.756060 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.755959    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69hgz\" (UniqueName: \"kubernetes.io/projected/7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b-kube-api-access-69hgz\") pod \"payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6\" (UID: \"7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"
Sep 15 11:38:09.814198 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.814161    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:09.857112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.856946    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3deab65e-55c8-4ca9-94a5-0e31678f10f9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb\" (UID: \"3deab65e-55c8-4ca9-94a5-0e31678f10f9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"
Sep 15 11:38:09.857112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.856993    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fc54\" (UniqueName: \"kubernetes.io/projected/3deab65e-55c8-4ca9-94a5-0e31678f10f9-kube-api-access-2fc54\") pod \"payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb\" (UID: \"3deab65e-55c8-4ca9-94a5-0e31678f10f9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"
Sep 15 11:38:09.857112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.857042    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69hgz\" (UniqueName: \"kubernetes.io/projected/7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b-kube-api-access-69hgz\") pod \"payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6\" (UID: \"7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"
Sep 15 11:38:09.857432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.857116    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6\" (UID: \"7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"
Sep 15 11:38:09.858035 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.857991    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3deab65e-55c8-4ca9-94a5-0e31678f10f9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb\" (UID: \"3deab65e-55c8-4ca9-94a5-0e31678f10f9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"
Sep 15 11:38:09.858035 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.857992    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6\" (UID: \"7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"
Sep 15 11:38:09.866508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.866488    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fc54\" (UniqueName: \"kubernetes.io/projected/3deab65e-55c8-4ca9-94a5-0e31678f10f9-kube-api-access-2fc54\") pod \"payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb\" (UID: \"3deab65e-55c8-4ca9-94a5-0e31678f10f9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"
Sep 15 11:38:09.866834 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:09.866815    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69hgz\" (UniqueName: \"kubernetes.io/projected/7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b-kube-api-access-69hgz\") pod \"payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6\" (UID: \"7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"
Sep 15 11:38:10.006637 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.006607    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"
Sep 15 11:38:10.013382 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.013354    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:10.042359 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.042324    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"
Sep 15 11:38:10.109319 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.108933    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"]
Sep 15 11:38:10.113007 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:38:10.112292    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb53f9c3_1536_4739_aed4_49bc9814d1ec.slice/crio-1f58c9217da88283a19866cbb2a66f53bd642cdd287c4a6f08357215795c15c2 WatchSource:0}: Error finding container 1f58c9217da88283a19866cbb2a66f53bd642cdd287c4a6f08357215795c15c2: Status 404 returned error can't find the container with id 1f58c9217da88283a19866cbb2a66f53bd642cdd287c4a6f08357215795c15c2
Sep 15 11:38:10.113214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.113100    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:10.288846 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.288818    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"]
Sep 15 11:38:10.291780 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:38:10.291741    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c39dbcc_2e09_4f4d_98dc_575b0ba78f7b.slice/crio-3ae1eeaa021e7811c871f40d8e3174ac1bf9bc3b428ba352918471b55f0e3f75 WatchSource:0}: Error finding container 3ae1eeaa021e7811c871f40d8e3174ac1bf9bc3b428ba352918471b55f0e3f75: Status 404 returned error can't find the container with id 3ae1eeaa021e7811c871f40d8e3174ac1bf9bc3b428ba352918471b55f0e3f75
Sep 15 11:38:10.342145 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.342120    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"]
Sep 15 11:38:10.344229 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:38:10.344203    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3deab65e_55c8_4ca9_94a5_0e31678f10f9.slice/crio-a780d99ce5cfada2e1fd983df406e6ada699af65fd76b4a4ac9d86818141cb11 WatchSource:0}: Error finding container a780d99ce5cfada2e1fd983df406e6ada699af65fd76b4a4ac9d86818141cb11: Status 404 returned error can't find the container with id a780d99ce5cfada2e1fd983df406e6ada699af65fd76b4a4ac9d86818141cb11
Sep 15 11:38:10.402601 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.402526    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:10.538189 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.538157    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:10.639260 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.639149    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:10.682291 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.682215    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6" event={"ID":"7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b","Type":"ContainerStarted","Data":"7cdb0a99e29996a572d49480164bb2d450f051439468d3670a81aed2c82ee7bb"}
Sep 15 11:38:10.682291 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.682277    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6" event={"ID":"7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b","Type":"ContainerStarted","Data":"3ae1eeaa021e7811c871f40d8e3174ac1bf9bc3b428ba352918471b55f0e3f75"}
Sep 15 11:38:10.682586 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.682309    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"
Sep 15 11:38:10.684249 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.684112    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb" event={"ID":"3deab65e-55c8-4ca9-94a5-0e31678f10f9","Type":"ContainerStarted","Data":"f346e56b178540107a2107832897f571c519cb043220925ec8f8ecc9bb35d554"}
Sep 15 11:38:10.684249 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.684173    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb" event={"ID":"3deab65e-55c8-4ca9-94a5-0e31678f10f9","Type":"ContainerStarted","Data":"a780d99ce5cfada2e1fd983df406e6ada699af65fd76b4a4ac9d86818141cb11"}
Sep 15 11:38:10.684249 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.684206    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"
Sep 15 11:38:10.686226 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.686199    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz" event={"ID":"cb53f9c3-1536-4739-aed4-49bc9814d1ec","Type":"ContainerStarted","Data":"c0161f46712e02d741ad535ee440a1bc0b23210ba0a737ea394e275303785fe7"}
Sep 15 11:38:10.686352 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.686246    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz" event={"ID":"cb53f9c3-1536-4739-aed4-49bc9814d1ec","Type":"ContainerStarted","Data":"1f58c9217da88283a19866cbb2a66f53bd642cdd287c4a6f08357215795c15c2"}
Sep 15 11:38:10.686352 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.686274    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:10.708439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.708382    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6" podStartSLOduration=1.708360925 podStartE2EDuration="1.708360925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:38:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:38:10.705216362 +0000 UTC m=+792.197993513" watchObservedRunningTime="2026-09-15 11:38:10.708360925 +0000 UTC m=+792.201138078"
Sep 15 11:38:10.745152 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.745089    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb" podStartSLOduration=1.7450679180000002 podStartE2EDuration="1.745067918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:38:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:38:10.742820389 +0000 UTC m=+792.235597531" watchObservedRunningTime="2026-09-15 11:38:10.745067918 +0000 UTC m=+792.237845067"
Sep 15 11:38:10.768716 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:10.768661    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz" podStartSLOduration=2.377287844 podStartE2EDuration="2.768640855s" totalImagesPullingTime="391.353011ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:38:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:38:10.766858504 +0000 UTC m=+792.259635653" watchObservedRunningTime="2026-09-15 11:38:10.768640855 +0000 UTC m=+792.261418005"
Sep 15 11:38:11.143268 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:11.143240    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:11.243185 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:11.243150    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:11.345792 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:11.345753    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:11.680974 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:11.680933    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:11.869974 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:11.869938    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:11.938014 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:11.937924    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"]
Sep 15 11:38:11.941812 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:11.941787    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"
Sep 15 11:38:11.944855 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:11.944827    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-b8fbdd\""
Sep 15 11:38:11.945007 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:11.944827    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-b8fbdd-dockercfg-cx6hw\""
Sep 15 11:38:11.956419 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:11.956394    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"]
Sep 15 11:38:11.976175 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:11.976150    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2233a306-f57a-47aa-8c1f-3d3c5ae642d7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw\" (UID: \"2233a306-f57a-47aa-8c1f-3d3c5ae642d7\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"
Sep 15 11:38:11.976334 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:11.976279    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p77xt\" (UniqueName: \"kubernetes.io/projected/2233a306-f57a-47aa-8c1f-3d3c5ae642d7-kube-api-access-p77xt\") pod \"payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw\" (UID: \"2233a306-f57a-47aa-8c1f-3d3c5ae642d7\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"
Sep 15 11:38:12.077770 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:12.077736    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p77xt\" (UniqueName: \"kubernetes.io/projected/2233a306-f57a-47aa-8c1f-3d3c5ae642d7-kube-api-access-p77xt\") pod \"payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw\" (UID: \"2233a306-f57a-47aa-8c1f-3d3c5ae642d7\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"
Sep 15 11:38:12.077946 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:12.077811    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2233a306-f57a-47aa-8c1f-3d3c5ae642d7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw\" (UID: \"2233a306-f57a-47aa-8c1f-3d3c5ae642d7\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"
Sep 15 11:38:12.078529 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:12.078508    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2233a306-f57a-47aa-8c1f-3d3c5ae642d7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw\" (UID: \"2233a306-f57a-47aa-8c1f-3d3c5ae642d7\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"
Sep 15 11:38:12.085840 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:12.085807    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p77xt\" (UniqueName: \"kubernetes.io/projected/2233a306-f57a-47aa-8c1f-3d3c5ae642d7-kube-api-access-p77xt\") pod \"payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw\" (UID: \"2233a306-f57a-47aa-8c1f-3d3c5ae642d7\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"
Sep 15 11:38:12.196587 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:12.196498    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:12.253382 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:12.253356    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"
Sep 15 11:38:12.296232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:12.296193    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:12.543325 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:12.543295    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"]
Sep 15 11:38:12.546008 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:38:12.545981    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2233a306_f57a_47aa_8c1f_3d3c5ae642d7.slice/crio-82fdd8b44a2dd5921ed5924832c9187f5de94b67f79667fe4ff2ccadf7cad19e WatchSource:0}: Error finding container 82fdd8b44a2dd5921ed5924832c9187f5de94b67f79667fe4ff2ccadf7cad19e: Status 404 returned error can't find the container with id 82fdd8b44a2dd5921ed5924832c9187f5de94b67f79667fe4ff2ccadf7cad19e
Sep 15 11:38:12.700739 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:12.700706    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw" event={"ID":"2233a306-f57a-47aa-8c1f-3d3c5ae642d7","Type":"ContainerStarted","Data":"1f3449dd0df72d63f1e7c4bd2428e12e4dadb047fa85b4477fc2bf55fd734956"}
Sep 15 11:38:12.700971 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:12.700797    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"
Sep 15 11:38:12.700971 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:12.700822    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw" event={"ID":"2233a306-f57a-47aa-8c1f-3d3c5ae642d7","Type":"ContainerStarted","Data":"82fdd8b44a2dd5921ed5924832c9187f5de94b67f79667fe4ff2ccadf7cad19e"}
Sep 15 11:38:12.722852 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:12.722796    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw" podStartSLOduration=1.722781696 podStartE2EDuration="1.722781696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:38:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:38:12.720247603 +0000 UTC m=+794.213024750" watchObservedRunningTime="2026-09-15 11:38:12.722781696 +0000 UTC m=+794.215558844"
Sep 15 11:38:12.911547 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:12.911495    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:13.028336 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:13.028295    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:13.183870 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:13.183789    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:13.284459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:13.284423    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:13.562599 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:13.562482    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:13.719020 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:13.718984    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:14.487218 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:14.487181    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:14.600610 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:14.600575    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:16.697231 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:16.697199    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:38:16.756784 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:16.756745    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:16.863526 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:16.863488    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:17.210321 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:17.210238    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:17.447458 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:17.447423    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:17.732987 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:17.732951    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:17.840743 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:17.840703    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:17.941467 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:17.941425    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"]
Sep 15 11:38:17.948851 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:17.948820    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"
Sep 15 11:38:17.960406 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:17.960382    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-4d6dbb\""
Sep 15 11:38:17.967731 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:17.967709    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"]
Sep 15 11:38:18.052640 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:18.052092    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd77a9cd-6675-4093-92b0-f9fef8723a2c-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d\" (UID: \"cd77a9cd-6675-4093-92b0-f9fef8723a2c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"
Sep 15 11:38:18.052640 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:18.052155    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb598\" (UniqueName: \"kubernetes.io/projected/cd77a9cd-6675-4093-92b0-f9fef8723a2c-kube-api-access-hb598\") pod \"payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d\" (UID: \"cd77a9cd-6675-4093-92b0-f9fef8723a2c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"
Sep 15 11:38:18.156420 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:18.156355    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd77a9cd-6675-4093-92b0-f9fef8723a2c-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d\" (UID: \"cd77a9cd-6675-4093-92b0-f9fef8723a2c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"
Sep 15 11:38:18.156646 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:18.156432    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb598\" (UniqueName: \"kubernetes.io/projected/cd77a9cd-6675-4093-92b0-f9fef8723a2c-kube-api-access-hb598\") pod \"payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d\" (UID: \"cd77a9cd-6675-4093-92b0-f9fef8723a2c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"
Sep 15 11:38:18.158137 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:18.158117    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd77a9cd-6675-4093-92b0-f9fef8723a2c-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d\" (UID: \"cd77a9cd-6675-4093-92b0-f9fef8723a2c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"
Sep 15 11:38:18.226585 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:18.225920    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb598\" (UniqueName: \"kubernetes.io/projected/cd77a9cd-6675-4093-92b0-f9fef8723a2c-kube-api-access-hb598\") pod \"payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d\" (UID: \"cd77a9cd-6675-4093-92b0-f9fef8723a2c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"
Sep 15 11:38:18.262680 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:18.262631    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"
Sep 15 11:38:18.608652 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:18.608601    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"]
Sep 15 11:38:18.743268 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:18.743191    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d" event={"ID":"cd77a9cd-6675-4093-92b0-f9fef8723a2c","Type":"ContainerStarted","Data":"c92fcf0cfa81802a8cc0bc153e03320132611f502e6a239945d9dd05265ae2db"}
Sep 15 11:38:18.951186 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:18.951098    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:19.752273 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:19.752213    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d" event={"ID":"cd77a9cd-6675-4093-92b0-f9fef8723a2c","Type":"ContainerStarted","Data":"59fc8a4f998ae8d8cb2f167db1f3d0e69b45631833a2f226ca33f2250c38962b"}
Sep 15 11:38:19.752695 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:19.752676    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"
Sep 15 11:38:19.777904 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:19.777846    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d" podStartSLOduration=2.777832083 podStartE2EDuration="2.777832083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:38:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:38:19.774356855 +0000 UTC m=+801.267134005" watchObservedRunningTime="2026-09-15 11:38:19.777832083 +0000 UTC m=+801.270609208"
Sep 15 11:38:20.583054 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:20.583003    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:21.150829 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:21.150716    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:21.381049 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:21.381015    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:21.715595 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:21.713126    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"
Sep 15 11:38:21.718594 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:21.716072    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"
Sep 15 11:38:21.766586 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:21.752493    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:21.922723 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:21.922689    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:22.085801 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:22.085714    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:23.146001 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.145951    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"]
Sep 15 11:38:23.150386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.150359    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:23.156927 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.156900    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f193f9fb-serving-cert\""
Sep 15 11:38:23.169328 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.169269    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"]
Sep 15 11:38:23.204858 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.204722    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:23.221143 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.221109    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8928eb53-0220-445e-b77d-47a9442f693c-maas-api-tls\") pod \"maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:23.221307 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.221157    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8928eb53-0220-445e-b77d-47a9442f693c-metrics-certs\") pod \"maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:23.221362 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.221295    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8928eb53-0220-445e-b77d-47a9442f693c-openshift-service-ca\") pod \"maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:23.221362 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.221354    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svl59\" (UniqueName: \"kubernetes.io/projected/8928eb53-0220-445e-b77d-47a9442f693c-kube-api-access-svl59\") pod \"maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:23.300248 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.300164    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:23.322226 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.322192    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8928eb53-0220-445e-b77d-47a9442f693c-openshift-service-ca\") pod \"maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:23.322421 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.322248    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svl59\" (UniqueName: \"kubernetes.io/projected/8928eb53-0220-445e-b77d-47a9442f693c-kube-api-access-svl59\") pod \"maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:23.322421 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.322365    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8928eb53-0220-445e-b77d-47a9442f693c-maas-api-tls\") pod \"maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:23.322421 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.322394    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8928eb53-0220-445e-b77d-47a9442f693c-metrics-certs\") pod \"maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:23.322638 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:38:23.322573    2560 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-serving-cert: secret "maas-api-e2e-ait-f193f9fb-serving-cert" not found
Sep 15 11:38:23.322702 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:38:23.322646    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8928eb53-0220-445e-b77d-47a9442f693c-maas-api-tls podName:8928eb53-0220-445e-b77d-47a9442f693c nodeName:}" failed. No retries permitted until 2026-09-15 11:38:23.822622947 +0000 UTC m=+805.315400090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8928eb53-0220-445e-b77d-47a9442f693c-maas-api-tls") pod "maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx" (UID: "8928eb53-0220-445e-b77d-47a9442f693c") : secret "maas-api-e2e-ait-f193f9fb-serving-cert" not found
Sep 15 11:38:23.323040 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.323013    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8928eb53-0220-445e-b77d-47a9442f693c-openshift-service-ca\") pod \"maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:23.325111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.325079    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8928eb53-0220-445e-b77d-47a9442f693c-metrics-certs\") pod \"maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:23.331984 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.331958    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svl59\" (UniqueName: \"kubernetes.io/projected/8928eb53-0220-445e-b77d-47a9442f693c-kube-api-access-svl59\") pod \"maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:23.397949 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.397785    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:23.711733 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.711648    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"
Sep 15 11:38:23.829278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.827894    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8928eb53-0220-445e-b77d-47a9442f693c-maas-api-tls\") pod \"maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:23.840220 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:23.840156    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8928eb53-0220-445e-b77d-47a9442f693c-maas-api-tls\") pod \"maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:24.067965 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.067881    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:24.197230 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.197184    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:24.317281 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.317241    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:24.369105 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.369076    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"]
Sep 15 11:38:24.446018 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.445973    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"]
Sep 15 11:38:24.449908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.449886    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"
Sep 15 11:38:24.452192 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.452067    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f193f9fb\""
Sep 15 11:38:24.452326 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.452280    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f193f9fb-dockercfg-pn9kt\""
Sep 15 11:38:24.462808 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.462776    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:24.471430 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.471404    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"]
Sep 15 11:38:24.528400 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.528368    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"]
Sep 15 11:38:24.532024 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.532001    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"
Sep 15 11:38:24.538630 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.538401    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38bbcfba-2566-4004-82f4-ff9c9616dad7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2\" (UID: \"38bbcfba-2566-4004-82f4-ff9c9616dad7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"
Sep 15 11:38:24.538707 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.538683    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f5xv\" (UniqueName: \"kubernetes.io/projected/38bbcfba-2566-4004-82f4-ff9c9616dad7-kube-api-access-5f5xv\") pod \"payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2\" (UID: \"38bbcfba-2566-4004-82f4-ff9c9616dad7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"
Sep 15 11:38:24.554631 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.554602    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"]
Sep 15 11:38:24.597452 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.592949    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:24.640233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.640137    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5qnp\" (UniqueName: \"kubernetes.io/projected/60b0b2e6-9488-4703-9ff5-9eb2587c9e42-kube-api-access-v5qnp\") pod \"payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr\" (UID: \"60b0b2e6-9488-4703-9ff5-9eb2587c9e42\") " pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"
Sep 15 11:38:24.640233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.640182    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38bbcfba-2566-4004-82f4-ff9c9616dad7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2\" (UID: \"38bbcfba-2566-4004-82f4-ff9c9616dad7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"
Sep 15 11:38:24.640475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.640291    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60b0b2e6-9488-4703-9ff5-9eb2587c9e42-plugins-config-volume\") pod \"payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr\" (UID: \"60b0b2e6-9488-4703-9ff5-9eb2587c9e42\") " pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"
Sep 15 11:38:24.640475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.640419    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f5xv\" (UniqueName: \"kubernetes.io/projected/38bbcfba-2566-4004-82f4-ff9c9616dad7-kube-api-access-5f5xv\") pod \"payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2\" (UID: \"38bbcfba-2566-4004-82f4-ff9c9616dad7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"
Sep 15 11:38:24.640966 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.640943    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38bbcfba-2566-4004-82f4-ff9c9616dad7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2\" (UID: \"38bbcfba-2566-4004-82f4-ff9c9616dad7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"
Sep 15 11:38:24.659332 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.659299    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f5xv\" (UniqueName: \"kubernetes.io/projected/38bbcfba-2566-4004-82f4-ff9c9616dad7-kube-api-access-5f5xv\") pod \"payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2\" (UID: \"38bbcfba-2566-4004-82f4-ff9c9616dad7\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"
Sep 15 11:38:24.694746 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.694706    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:24.741588 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.741544    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60b0b2e6-9488-4703-9ff5-9eb2587c9e42-plugins-config-volume\") pod \"payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr\" (UID: \"60b0b2e6-9488-4703-9ff5-9eb2587c9e42\") " pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"
Sep 15 11:38:24.741974 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.741939    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5qnp\" (UniqueName: \"kubernetes.io/projected/60b0b2e6-9488-4703-9ff5-9eb2587c9e42-kube-api-access-v5qnp\") pod \"payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr\" (UID: \"60b0b2e6-9488-4703-9ff5-9eb2587c9e42\") " pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"
Sep 15 11:38:24.742343 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.742313    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60b0b2e6-9488-4703-9ff5-9eb2587c9e42-plugins-config-volume\") pod \"payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr\" (UID: \"60b0b2e6-9488-4703-9ff5-9eb2587c9e42\") " pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"
Sep 15 11:38:24.752034 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.752006    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5qnp\" (UniqueName: \"kubernetes.io/projected/60b0b2e6-9488-4703-9ff5-9eb2587c9e42-kube-api-access-v5qnp\") pod \"payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr\" (UID: \"60b0b2e6-9488-4703-9ff5-9eb2587c9e42\") " pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"
Sep 15 11:38:24.761640 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.761619    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"
Sep 15 11:38:24.809777 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.809743    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx" event={"ID":"8928eb53-0220-445e-b77d-47a9442f693c","Type":"ContainerStarted","Data":"fe9372ea7852a0150615bf378ac5ead772782e85ba092f91cebb75adea1b4a09"}
Sep 15 11:38:24.843501 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:24.843191    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"
Sep 15 11:38:25.008844 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:25.008809    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:25.058483 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:25.058455    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"]
Sep 15 11:38:25.061650 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:38:25.061619    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38bbcfba_2566_4004_82f4_ff9c9616dad7.slice/crio-60e1b19801809596d673c54ecd4c9386d668eaa487cda40238c9d6e1199131fe WatchSource:0}: Error finding container 60e1b19801809596d673c54ecd4c9386d668eaa487cda40238c9d6e1199131fe: Status 404 returned error can't find the container with id 60e1b19801809596d673c54ecd4c9386d668eaa487cda40238c9d6e1199131fe
Sep 15 11:38:25.148612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:25.148587    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"]
Sep 15 11:38:25.150895 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:38:25.150861    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60b0b2e6_9488_4703_9ff5_9eb2587c9e42.slice/crio-33ccc7fd91a11f78eb79952a25030af8c152af767c9b849aa19619c011d403e5 WatchSource:0}: Error finding container 33ccc7fd91a11f78eb79952a25030af8c152af767c9b849aa19619c011d403e5: Status 404 returned error can't find the container with id 33ccc7fd91a11f78eb79952a25030af8c152af767c9b849aa19619c011d403e5
Sep 15 11:38:25.823149 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:25.820521    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2" event={"ID":"38bbcfba-2566-4004-82f4-ff9c9616dad7","Type":"ContainerStarted","Data":"672a725c09b20cf387aa5eb3615eb71fa9d1096deeda6f5c6db4254714983b41"}
Sep 15 11:38:25.823149 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:25.820690    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2" event={"ID":"38bbcfba-2566-4004-82f4-ff9c9616dad7","Type":"ContainerStarted","Data":"60e1b19801809596d673c54ecd4c9386d668eaa487cda40238c9d6e1199131fe"}
Sep 15 11:38:25.823149 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:25.821232    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"
Sep 15 11:38:25.823149 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:25.822218    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr" event={"ID":"60b0b2e6-9488-4703-9ff5-9eb2587c9e42","Type":"ContainerStarted","Data":"8b13f42087b31182566e550609002e2ddc93477a5e62b34747cba90fd5a4164a"}
Sep 15 11:38:25.823149 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:25.822266    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr" event={"ID":"60b0b2e6-9488-4703-9ff5-9eb2587c9e42","Type":"ContainerStarted","Data":"33ccc7fd91a11f78eb79952a25030af8c152af767c9b849aa19619c011d403e5"}
Sep 15 11:38:25.823149 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:25.822341    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"
Sep 15 11:38:25.849049 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:25.823829    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx" event={"ID":"8928eb53-0220-445e-b77d-47a9442f693c","Type":"ContainerStarted","Data":"61be85baa7b4bd83283c7794c1a4e5e2619bd36b4d156d7b7b9869a22018092c"}
Sep 15 11:38:25.849049 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:25.823954    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:25.849049 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:25.841647    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2" podStartSLOduration=1.8416309260000001 podStartE2EDuration="1.841630926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:38:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:38:25.839965307 +0000 UTC m=+807.332742457" watchObservedRunningTime="2026-09-15 11:38:25.841630926 +0000 UTC m=+807.334408075"
Sep 15 11:38:25.858135 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:25.858095    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr" podStartSLOduration=1.858081652 podStartE2EDuration="1.858081652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:38:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:38:25.856910278 +0000 UTC m=+807.349687427" watchObservedRunningTime="2026-09-15 11:38:25.858081652 +0000 UTC m=+807.350858800"
Sep 15 11:38:25.877575 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:25.877521    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx" podStartSLOduration=2.53708242 podStartE2EDuration="2.877506449s" totalImagesPullingTime="340.424029ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:38:25.876543853 +0000 UTC m=+807.369321001" watchObservedRunningTime="2026-09-15 11:38:25.877506449 +0000 UTC m=+807.370283598"
Sep 15 11:38:26.080852 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.080774    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:26.566642 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.566604    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:26.666478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.666446    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:26.831621 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.831531    2560 generic.go:356] "Generic (PLEG): container finished" podID="8f737c66-5417-491d-8c79-9a4649e2eb7a" containerID="023166f619ab846f2d9d82fbc15449a5a5fd741cb302ca251c60e1de04c7b612" exitCode=0
Sep 15 11:38:26.832049 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.831616    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz" event={"ID":"8f737c66-5417-491d-8c79-9a4649e2eb7a","Type":"ContainerDied","Data":"023166f619ab846f2d9d82fbc15449a5a5fd741cb302ca251c60e1de04c7b612"}
Sep 15 11:38:26.832174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.832156    2560 scope.go:117] "RemoveContainer" containerID="023166f619ab846f2d9d82fbc15449a5a5fd741cb302ca251c60e1de04c7b612"
Sep 15 11:38:26.833543 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.833504    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:38:26.833691 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.833618    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:38:26.833691 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.833667    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:38:26.833830 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.833804    2560 generic.go:356] "Generic (PLEG): container finished" podID="0582ae44-03a9-4a95-841d-ef551a58fc19" containerID="ae364f9002995116d5d754ab15b26327b1225fe850b5c6ac87ee9cff0207d797" exitCode=0
Sep 15 11:38:26.833895 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.833875    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4" event={"ID":"0582ae44-03a9-4a95-841d-ef551a58fc19","Type":"ContainerDied","Data":"ae364f9002995116d5d754ab15b26327b1225fe850b5c6ac87ee9cff0207d797"}
Sep 15 11:38:26.834399 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.834379    2560 scope.go:117] "RemoveContainer" containerID="ae364f9002995116d5d754ab15b26327b1225fe850b5c6ac87ee9cff0207d797"
Sep 15 11:38:26.835768 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.835736    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:38:26.835859 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.835815    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:38:26.835901 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.835860    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:38:26.966146 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:26.964861    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:27.165346 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:27.165183    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:27.436790 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:27.436691    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:27.722725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:27.722647    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:27.841930 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:27.841888    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz" event={"ID":"8f737c66-5417-491d-8c79-9a4649e2eb7a","Type":"ContainerStarted","Data":"49790447d4ebe38548c046d8419ce2c636f7051005e7fd9bd2a99583ee909d77"}
Sep 15 11:38:27.844079 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:27.844044    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4" event={"ID":"0582ae44-03a9-4a95-841d-ef551a58fc19","Type":"ContainerStarted","Data":"01f1129c11173137862446600fcecbf7783e3fd28646d0c5fc530afd97af1103"}
Sep 15 11:38:28.703234 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:28.703188    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:28.703682 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:28.703646    2560 patch_prober.go:28] interesting pod/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 15 11:38:28.703821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:28.703709    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4" podUID="0582ae44-03a9-4a95-841d-ef551a58fc19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 15 11:38:28.778233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:28.778193    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:28.812217 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:28.812180    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:28.812878 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:28.812771    2560 patch_prober.go:28] interesting pod/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 15 11:38:28.813095 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:28.813071    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz" podUID="8f737c66-5417-491d-8c79-9a4649e2eb7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 15 11:38:29.132257 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:29.132216    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:29.703751 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:29.703710    2560 patch_prober.go:28] interesting pod/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 15 11:38:29.704004 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:29.703778    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4" podUID="0582ae44-03a9-4a95-841d-ef551a58fc19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 15 11:38:29.812139 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:29.812083    2560 patch_prober.go:28] interesting pod/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 15 11:38:29.812343 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:29.812174    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz" podUID="8f737c66-5417-491d-8c79-9a4649e2eb7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 15 11:38:29.820265 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:29.820214    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:30.080367 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:30.080277    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:30.703667 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:30.703629    2560 patch_prober.go:28] interesting pod/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 15 11:38:30.704265 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:30.704231    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4" podUID="0582ae44-03a9-4a95-841d-ef551a58fc19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 15 11:38:30.763368 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:30.763337    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"
Sep 15 11:38:30.825585 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:30.824007    2560 patch_prober.go:28] interesting pod/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 15 11:38:30.825585 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:30.824071    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz" podUID="8f737c66-5417-491d-8c79-9a4649e2eb7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 15 11:38:30.865037 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:30.865003    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:31.019996 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.019904    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:31.612262 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.612220    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:31.700893 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.700859    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:31.703442 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.703412    2560 patch_prober.go:28] interesting pod/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 15 11:38:31.703676 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.703645    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4" podUID="0582ae44-03a9-4a95-841d-ef551a58fc19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 15 11:38:31.783600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.783549    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"]
Sep 15 11:38:31.808898 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.808862    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"
Sep 15 11:38:31.810984 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.810953    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"]
Sep 15 11:38:31.818691 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.818658    2560 patch_prober.go:28] interesting pod/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 15 11:38:31.818818 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.818725    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz" podUID="8f737c66-5417-491d-8c79-9a4649e2eb7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 15 11:38:31.819553 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.819526    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-78522e-dockercfg-8m9q7\""
Sep 15 11:38:31.819849 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.819828    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-78522e\""
Sep 15 11:38:31.865945 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.865861    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:31.949595 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.947699    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7b2de0a-ba15-4ea9-908a-4767e24e934b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf\" (UID: \"e7b2de0a-ba15-4ea9-908a-4767e24e934b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"
Sep 15 11:38:31.949595 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.947861    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkx6r\" (UniqueName: \"kubernetes.io/projected/e7b2de0a-ba15-4ea9-908a-4767e24e934b-kube-api-access-qkx6r\") pod \"payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf\" (UID: \"e7b2de0a-ba15-4ea9-908a-4767e24e934b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"
Sep 15 11:38:31.968640 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:31.968602    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:32.049116 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.049081    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkx6r\" (UniqueName: \"kubernetes.io/projected/e7b2de0a-ba15-4ea9-908a-4767e24e934b-kube-api-access-qkx6r\") pod \"payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf\" (UID: \"e7b2de0a-ba15-4ea9-908a-4767e24e934b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"
Sep 15 11:38:32.049401 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.049386    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7b2de0a-ba15-4ea9-908a-4767e24e934b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf\" (UID: \"e7b2de0a-ba15-4ea9-908a-4767e24e934b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"
Sep 15 11:38:32.051245 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.051180    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7b2de0a-ba15-4ea9-908a-4767e24e934b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf\" (UID: \"e7b2de0a-ba15-4ea9-908a-4767e24e934b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"
Sep 15 11:38:32.071966 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.071850    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkx6r\" (UniqueName: \"kubernetes.io/projected/e7b2de0a-ba15-4ea9-908a-4767e24e934b-kube-api-access-qkx6r\") pod \"payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf\" (UID: \"e7b2de0a-ba15-4ea9-908a-4767e24e934b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"
Sep 15 11:38:32.109257 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.109217    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:32.157552 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.157467    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"
Sep 15 11:38:32.230585 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.229705    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:32.371349 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.371315    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:32.478906 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.478825    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:32.495401 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.494392    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"]
Sep 15 11:38:32.512158 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:38:32.512121    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7b2de0a_ba15_4ea9_908a_4767e24e934b.slice/crio-052228f3b68412545e4dc3be20cc20205c391e6f91a60408f598a6dda8c06257 WatchSource:0}: Error finding container 052228f3b68412545e4dc3be20cc20205c391e6f91a60408f598a6dda8c06257: Status 404 returned error can't find the container with id 052228f3b68412545e4dc3be20cc20205c391e6f91a60408f598a6dda8c06257
Sep 15 11:38:32.703649 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.703190    2560 patch_prober.go:28] interesting pod/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Sep 15 11:38:32.703649 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.703255    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4" podUID="0582ae44-03a9-4a95-841d-ef551a58fc19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Sep 15 11:38:32.812611 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.812502    2560 patch_prober.go:28] interesting pod/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 15 11:38:32.813261 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.813158    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz" podUID="8f737c66-5417-491d-8c79-9a4649e2eb7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 15 11:38:32.879676 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.879632    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf" event={"ID":"e7b2de0a-ba15-4ea9-908a-4767e24e934b","Type":"ContainerStarted","Data":"7b504c078a99f39bedd46ad9ec95a32a169a2dae4124c1e5a95d8cb539b895a6"}
Sep 15 11:38:32.879860 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.879696    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf" event={"ID":"e7b2de0a-ba15-4ea9-908a-4767e24e934b","Type":"ContainerStarted","Data":"052228f3b68412545e4dc3be20cc20205c391e6f91a60408f598a6dda8c06257"}
Sep 15 11:38:32.879860 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:32.879749    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"
Sep 15 11:38:33.699043 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:33.699009    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:33.801165 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:33.801131    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:33.801604 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:33.801550    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:33.802907 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:33.802883    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:38:33.821181 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:33.821144    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:33.821748 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:33.821728    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:33.825461 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:33.825432    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:38:33.826648 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:33.826596    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf" podStartSLOduration=2.826579053 podStartE2EDuration="2.826579053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:38:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:38:32.913586214 +0000 UTC m=+814.406363355" watchObservedRunningTime="2026-09-15 11:38:33.826579053 +0000 UTC m=+815.319356216"
Sep 15 11:38:34.814228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:34.814191    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:35.429245 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:35.429208    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:35.680924 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:35.680840    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:36.491898 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:36.491855    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:36.835858 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:36.835772    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"
Sep 15 11:38:37.281191 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:37.281151    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:37.846717 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:37.846686    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"
Sep 15 11:38:38.173470 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:38.173387    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:39.423278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:39.423244    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:40.218243 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:40.218198    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:40.316196 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:40.316162    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:40.435821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:40.435789    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:41.331106 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:41.331073    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:42.326955 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:42.326920    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:42.856072 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:42.856036    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:43.227968 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:43.227864    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:43.892156 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:43.892124    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"
Sep 15 11:38:44.282033 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:44.281946    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:45.159341 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:45.159310    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:45.981512 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:45.981465    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:46.772049 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:46.772006    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:47.731206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:47.731169    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:48.428662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:48.426649    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:49.353206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:49.353167    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:49.455328 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:49.455287    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:50.136543 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:50.136490    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:51.078630 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.078595    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"]
Sep 15 11:38:51.079073 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.078791    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx" podUID="8928eb53-0220-445e-b77d-47a9442f693c" containerName="maas-api" containerID="cri-o://61be85baa7b4bd83283c7794c1a4e5e2619bd36b4d156d7b7b9869a22018092c" gracePeriod=30
Sep 15 11:38:51.120279 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.120250    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:51.229648 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.229605    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"]
Sep 15 11:38:51.229969 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.229898    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2" podUID="38bbcfba-2566-4004-82f4-ff9c9616dad7" containerName="payload-pre-processing" containerID="cri-o://672a725c09b20cf387aa5eb3615eb71fa9d1096deeda6f5c6db4254714983b41" gracePeriod=30
Sep 15 11:38:51.234659 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.234633    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"]
Sep 15 11:38:51.235293 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.235226    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr" podUID="60b0b2e6-9488-4703-9ff5-9eb2587c9e42" containerName="payload-processing" containerID="cri-o://8b13f42087b31182566e550609002e2ddc93477a5e62b34747cba90fd5a4164a" gracePeriod=30
Sep 15 11:38:51.270937 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.270880    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:51.369440 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.369403    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:51.379821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.379693    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:51.442848 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.442829    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8928eb53-0220-445e-b77d-47a9442f693c-metrics-certs\") pod \"8928eb53-0220-445e-b77d-47a9442f693c\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") "
Sep 15 11:38:51.442928 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.442874    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8928eb53-0220-445e-b77d-47a9442f693c-openshift-service-ca\") pod \"8928eb53-0220-445e-b77d-47a9442f693c\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") "
Sep 15 11:38:51.443050 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.443016    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8928eb53-0220-445e-b77d-47a9442f693c-maas-api-tls\") pod \"8928eb53-0220-445e-b77d-47a9442f693c\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") "
Sep 15 11:38:51.443120 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.443067    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svl59\" (UniqueName: \"kubernetes.io/projected/8928eb53-0220-445e-b77d-47a9442f693c-kube-api-access-svl59\") pod \"8928eb53-0220-445e-b77d-47a9442f693c\" (UID: \"8928eb53-0220-445e-b77d-47a9442f693c\") "
Sep 15 11:38:51.443445 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.443331    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8928eb53-0220-445e-b77d-47a9442f693c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8928eb53-0220-445e-b77d-47a9442f693c" (UID: "8928eb53-0220-445e-b77d-47a9442f693c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:38:51.443445 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.443442    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8928eb53-0220-445e-b77d-47a9442f693c-openshift-service-ca\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:38:51.447260 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.447064    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8928eb53-0220-445e-b77d-47a9442f693c-kube-api-access-svl59" (OuterVolumeSpecName: "kube-api-access-svl59") pod "8928eb53-0220-445e-b77d-47a9442f693c" (UID: "8928eb53-0220-445e-b77d-47a9442f693c"). InnerVolumeSpecName "kube-api-access-svl59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:38:51.447260 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.447222    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8928eb53-0220-445e-b77d-47a9442f693c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8928eb53-0220-445e-b77d-47a9442f693c" (UID: "8928eb53-0220-445e-b77d-47a9442f693c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:38:51.447720 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.447458    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8928eb53-0220-445e-b77d-47a9442f693c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8928eb53-0220-445e-b77d-47a9442f693c" (UID: "8928eb53-0220-445e-b77d-47a9442f693c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:38:51.488481 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.488331    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:51.508725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.508693    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"
Sep 15 11:38:51.521924 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.521904    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"
Sep 15 11:38:51.545035 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.545006    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8928eb53-0220-445e-b77d-47a9442f693c-metrics-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:38:51.545035 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.545037    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8928eb53-0220-445e-b77d-47a9442f693c-maas-api-tls\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:38:51.545227 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.545052    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svl59\" (UniqueName: \"kubernetes.io/projected/8928eb53-0220-445e-b77d-47a9442f693c-kube-api-access-svl59\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:38:51.591959 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.591751    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:51.646297 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.646269    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38bbcfba-2566-4004-82f4-ff9c9616dad7-plugins-config-volume\") pod \"38bbcfba-2566-4004-82f4-ff9c9616dad7\" (UID: \"38bbcfba-2566-4004-82f4-ff9c9616dad7\") "
Sep 15 11:38:51.646488 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.646331    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5qnp\" (UniqueName: \"kubernetes.io/projected/60b0b2e6-9488-4703-9ff5-9eb2587c9e42-kube-api-access-v5qnp\") pod \"60b0b2e6-9488-4703-9ff5-9eb2587c9e42\" (UID: \"60b0b2e6-9488-4703-9ff5-9eb2587c9e42\") "
Sep 15 11:38:51.646488 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.646418    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60b0b2e6-9488-4703-9ff5-9eb2587c9e42-plugins-config-volume\") pod \"60b0b2e6-9488-4703-9ff5-9eb2587c9e42\" (UID: \"60b0b2e6-9488-4703-9ff5-9eb2587c9e42\") "
Sep 15 11:38:51.646488 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.646462    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5f5xv\" (UniqueName: \"kubernetes.io/projected/38bbcfba-2566-4004-82f4-ff9c9616dad7-kube-api-access-5f5xv\") pod \"38bbcfba-2566-4004-82f4-ff9c9616dad7\" (UID: \"38bbcfba-2566-4004-82f4-ff9c9616dad7\") "
Sep 15 11:38:51.646754 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.646720    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38bbcfba-2566-4004-82f4-ff9c9616dad7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "38bbcfba-2566-4004-82f4-ff9c9616dad7" (UID: "38bbcfba-2566-4004-82f4-ff9c9616dad7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:38:51.646850 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.646778    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60b0b2e6-9488-4703-9ff5-9eb2587c9e42-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "60b0b2e6-9488-4703-9ff5-9eb2587c9e42" (UID: "60b0b2e6-9488-4703-9ff5-9eb2587c9e42"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:38:51.646912 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.646865    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38bbcfba-2566-4004-82f4-ff9c9616dad7-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:38:51.648621 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.648593    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60b0b2e6-9488-4703-9ff5-9eb2587c9e42-kube-api-access-v5qnp" (OuterVolumeSpecName: "kube-api-access-v5qnp") pod "60b0b2e6-9488-4703-9ff5-9eb2587c9e42" (UID: "60b0b2e6-9488-4703-9ff5-9eb2587c9e42"). InnerVolumeSpecName "kube-api-access-v5qnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:38:51.648742 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.648712    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38bbcfba-2566-4004-82f4-ff9c9616dad7-kube-api-access-5f5xv" (OuterVolumeSpecName: "kube-api-access-5f5xv") pod "38bbcfba-2566-4004-82f4-ff9c9616dad7" (UID: "38bbcfba-2566-4004-82f4-ff9c9616dad7"). InnerVolumeSpecName "kube-api-access-5f5xv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:38:51.683929 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.683903    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:51.747829 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.747794    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5f5xv\" (UniqueName: \"kubernetes.io/projected/38bbcfba-2566-4004-82f4-ff9c9616dad7-kube-api-access-5f5xv\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:38:51.747829 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.747826    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5qnp\" (UniqueName: \"kubernetes.io/projected/60b0b2e6-9488-4703-9ff5-9eb2587c9e42-kube-api-access-v5qnp\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:38:51.747829 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:51.747835    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60b0b2e6-9488-4703-9ff5-9eb2587c9e42-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:38:52.006299 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.006221    2560 generic.go:356] "Generic (PLEG): container finished" podID="38bbcfba-2566-4004-82f4-ff9c9616dad7" containerID="672a725c09b20cf387aa5eb3615eb71fa9d1096deeda6f5c6db4254714983b41" exitCode=0
Sep 15 11:38:52.006299 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.006262    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"
Sep 15 11:38:52.006495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.006304    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2" event={"ID":"38bbcfba-2566-4004-82f4-ff9c9616dad7","Type":"ContainerDied","Data":"672a725c09b20cf387aa5eb3615eb71fa9d1096deeda6f5c6db4254714983b41"}
Sep 15 11:38:52.006495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.006369    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2" event={"ID":"38bbcfba-2566-4004-82f4-ff9c9616dad7","Type":"ContainerDied","Data":"60e1b19801809596d673c54ecd4c9386d668eaa487cda40238c9d6e1199131fe"}
Sep 15 11:38:52.006495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.006403    2560 scope.go:117] "RemoveContainer" containerID="672a725c09b20cf387aa5eb3615eb71fa9d1096deeda6f5c6db4254714983b41"
Sep 15 11:38:52.007694 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.007672    2560 generic.go:356] "Generic (PLEG): container finished" podID="60b0b2e6-9488-4703-9ff5-9eb2587c9e42" containerID="8b13f42087b31182566e550609002e2ddc93477a5e62b34747cba90fd5a4164a" exitCode=0
Sep 15 11:38:52.007764 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.007716    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"
Sep 15 11:38:52.007822 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.007752    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr" event={"ID":"60b0b2e6-9488-4703-9ff5-9eb2587c9e42","Type":"ContainerDied","Data":"8b13f42087b31182566e550609002e2ddc93477a5e62b34747cba90fd5a4164a"}
Sep 15 11:38:52.007822 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.007813    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr" event={"ID":"60b0b2e6-9488-4703-9ff5-9eb2587c9e42","Type":"ContainerDied","Data":"33ccc7fd91a11f78eb79952a25030af8c152af767c9b849aa19619c011d403e5"}
Sep 15 11:38:52.009333 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.009308    2560 generic.go:356] "Generic (PLEG): container finished" podID="8928eb53-0220-445e-b77d-47a9442f693c" containerID="61be85baa7b4bd83283c7794c1a4e5e2619bd36b4d156d7b7b9869a22018092c" exitCode=0
Sep 15 11:38:52.009478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.009342    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"
Sep 15 11:38:52.009478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.009341    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx" event={"ID":"8928eb53-0220-445e-b77d-47a9442f693c","Type":"ContainerDied","Data":"61be85baa7b4bd83283c7794c1a4e5e2619bd36b4d156d7b7b9869a22018092c"}
Sep 15 11:38:52.009478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.009469    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx" event={"ID":"8928eb53-0220-445e-b77d-47a9442f693c","Type":"ContainerDied","Data":"fe9372ea7852a0150615bf378ac5ead772782e85ba092f91cebb75adea1b4a09"}
Sep 15 11:38:52.017696 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.017679    2560 scope.go:117] "RemoveContainer" containerID="672a725c09b20cf387aa5eb3615eb71fa9d1096deeda6f5c6db4254714983b41"
Sep 15 11:38:52.017972 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:38:52.017953    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"672a725c09b20cf387aa5eb3615eb71fa9d1096deeda6f5c6db4254714983b41\": container with ID starting with 672a725c09b20cf387aa5eb3615eb71fa9d1096deeda6f5c6db4254714983b41 not found: ID does not exist" containerID="672a725c09b20cf387aa5eb3615eb71fa9d1096deeda6f5c6db4254714983b41"
Sep 15 11:38:52.018047 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.017990    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"672a725c09b20cf387aa5eb3615eb71fa9d1096deeda6f5c6db4254714983b41"} err="failed to get container status \"672a725c09b20cf387aa5eb3615eb71fa9d1096deeda6f5c6db4254714983b41\": rpc error: code = NotFound desc = could not find container \"672a725c09b20cf387aa5eb3615eb71fa9d1096deeda6f5c6db4254714983b41\": container with ID starting with 672a725c09b20cf387aa5eb3615eb71fa9d1096deeda6f5c6db4254714983b41 not found: ID does not exist"
Sep 15 11:38:52.018047 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.018012    2560 scope.go:117] "RemoveContainer" containerID="8b13f42087b31182566e550609002e2ddc93477a5e62b34747cba90fd5a4164a"
Sep 15 11:38:52.026493 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.026459    2560 scope.go:117] "RemoveContainer" containerID="8b13f42087b31182566e550609002e2ddc93477a5e62b34747cba90fd5a4164a"
Sep 15 11:38:52.026752 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:38:52.026734    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b13f42087b31182566e550609002e2ddc93477a5e62b34747cba90fd5a4164a\": container with ID starting with 8b13f42087b31182566e550609002e2ddc93477a5e62b34747cba90fd5a4164a not found: ID does not exist" containerID="8b13f42087b31182566e550609002e2ddc93477a5e62b34747cba90fd5a4164a"
Sep 15 11:38:52.026814 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.026759    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b13f42087b31182566e550609002e2ddc93477a5e62b34747cba90fd5a4164a"} err="failed to get container status \"8b13f42087b31182566e550609002e2ddc93477a5e62b34747cba90fd5a4164a\": rpc error: code = NotFound desc = could not find container \"8b13f42087b31182566e550609002e2ddc93477a5e62b34747cba90fd5a4164a\": container with ID starting with 8b13f42087b31182566e550609002e2ddc93477a5e62b34747cba90fd5a4164a not found: ID does not exist"
Sep 15 11:38:52.026814 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.026775    2560 scope.go:117] "RemoveContainer" containerID="61be85baa7b4bd83283c7794c1a4e5e2619bd36b4d156d7b7b9869a22018092c"
Sep 15 11:38:52.034368 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.034352    2560 scope.go:117] "RemoveContainer" containerID="61be85baa7b4bd83283c7794c1a4e5e2619bd36b4d156d7b7b9869a22018092c"
Sep 15 11:38:52.034647 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:38:52.034621    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61be85baa7b4bd83283c7794c1a4e5e2619bd36b4d156d7b7b9869a22018092c\": container with ID starting with 61be85baa7b4bd83283c7794c1a4e5e2619bd36b4d156d7b7b9869a22018092c not found: ID does not exist" containerID="61be85baa7b4bd83283c7794c1a4e5e2619bd36b4d156d7b7b9869a22018092c"
Sep 15 11:38:52.034741 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.034653    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61be85baa7b4bd83283c7794c1a4e5e2619bd36b4d156d7b7b9869a22018092c"} err="failed to get container status \"61be85baa7b4bd83283c7794c1a4e5e2619bd36b4d156d7b7b9869a22018092c\": rpc error: code = NotFound desc = could not find container \"61be85baa7b4bd83283c7794c1a4e5e2619bd36b4d156d7b7b9869a22018092c\": container with ID starting with 61be85baa7b4bd83283c7794c1a4e5e2619bd36b4d156d7b7b9869a22018092c not found: ID does not exist"
Sep 15 11:38:52.043084 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.043062    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"]
Sep 15 11:38:52.047515 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.047495    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f193f9fb-786c8bb478-vjmw2"]
Sep 15 11:38:52.068618 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.068597    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"]
Sep 15 11:38:52.077097 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.077075    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f193f9fb-746ddf768c-knvkr"]
Sep 15 11:38:52.105681 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.105647    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"]
Sep 15 11:38:52.119062 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.119038    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f193f9fb-8569595dbc-rfhnx"]
Sep 15 11:38:52.160831 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.160806    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:52.262226 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.262138    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:52.367269 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.367238    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:52.632371 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.632342    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"]
Sep 15 11:38:52.632817 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.632801    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38bbcfba-2566-4004-82f4-ff9c9616dad7" containerName="payload-pre-processing"
Sep 15 11:38:52.632880 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.632818    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38bbcfba-2566-4004-82f4-ff9c9616dad7" containerName="payload-pre-processing"
Sep 15 11:38:52.632880 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.632826    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60b0b2e6-9488-4703-9ff5-9eb2587c9e42" containerName="payload-processing"
Sep 15 11:38:52.632880 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.632832    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="60b0b2e6-9488-4703-9ff5-9eb2587c9e42" containerName="payload-processing"
Sep 15 11:38:52.632880 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.632840    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8928eb53-0220-445e-b77d-47a9442f693c" containerName="maas-api"
Sep 15 11:38:52.632880 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.632846    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8928eb53-0220-445e-b77d-47a9442f693c" containerName="maas-api"
Sep 15 11:38:52.633041 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.632930    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8928eb53-0220-445e-b77d-47a9442f693c" containerName="maas-api"
Sep 15 11:38:52.633041 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.632940    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="38bbcfba-2566-4004-82f4-ff9c9616dad7" containerName="payload-pre-processing"
Sep 15 11:38:52.633041 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.632947    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="60b0b2e6-9488-4703-9ff5-9eb2587c9e42" containerName="payload-processing"
Sep 15 11:38:52.637611 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.637589    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.640354 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.640336    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-07a6c076-gw-openshift-default-dockercfg-dx5r5\""
Sep 15 11:38:52.653866 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.653839    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"]
Sep 15 11:38:52.675993 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.675959    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:52.758555 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.757026    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5375fb3-7899-4ad2-965d-61707dac6c93-istiod-ca-cert\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.758555 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.757114    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-workload-certs\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.758555 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.757193    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-workload-socket\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.758555 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.757217    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-data\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.758555 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.757249    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-token\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.758555 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.757287    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-envoy\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.758555 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.757334    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-credential-socket\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.758555 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.757437    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwjcz\" (UniqueName: \"kubernetes.io/projected/b5375fb3-7899-4ad2-965d-61707dac6c93-kube-api-access-zwjcz\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.758555 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.757477    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-podinfo\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.774086 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.774029    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:52.858477 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.858443    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwjcz\" (UniqueName: \"kubernetes.io/projected/b5375fb3-7899-4ad2-965d-61707dac6c93-kube-api-access-zwjcz\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.858682 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.858487    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-podinfo\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.858682 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.858655    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5375fb3-7899-4ad2-965d-61707dac6c93-istiod-ca-cert\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.858828 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.858741    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-workload-certs\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.858828 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.858810    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-workload-socket\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.858936 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.858829    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-data\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.858936 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.858849    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-token\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.858936 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.858882    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-envoy\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.859109 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.859088    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-credential-socket\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.859161 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.859125    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-workload-certs\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.859309 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.859287    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-workload-socket\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.859434 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.859409    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5375fb3-7899-4ad2-965d-61707dac6c93-istiod-ca-cert\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.859589 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.859414    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-data\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.859589 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.859550    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-credential-socket\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.860999 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.860977    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-envoy\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.861073 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.860980    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-podinfo\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.870084 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.870058    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-token\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.870910 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.870891    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwjcz\" (UniqueName: \"kubernetes.io/projected/b5375fb3-7899-4ad2-965d-61707dac6c93-kube-api-access-zwjcz\") pod \"e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") " pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:52.948177 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:52.948111    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:53.035293 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:53.035261    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38bbcfba-2566-4004-82f4-ff9c9616dad7" path="/var/lib/kubelet/pods/38bbcfba-2566-4004-82f4-ff9c9616dad7/volumes"
Sep 15 11:38:53.036182 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:53.036159    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60b0b2e6-9488-4703-9ff5-9eb2587c9e42" path="/var/lib/kubelet/pods/60b0b2e6-9488-4703-9ff5-9eb2587c9e42/volumes"
Sep 15 11:38:53.036737 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:53.036714    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8928eb53-0220-445e-b77d-47a9442f693c" path="/var/lib/kubelet/pods/8928eb53-0220-445e-b77d-47a9442f693c/volumes"
Sep 15 11:38:53.114860 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:53.114824    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:53.231509 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:53.231486    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"]
Sep 15 11:38:53.233260 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:38:53.233232    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5375fb3_7899_4ad2_965d_61707dac6c93.slice/crio-3fb4fd82b49743abc2bb1989da0ec1b78a1dfbea14bbeb1d7dac02084bc7c194 WatchSource:0}: Error finding container 3fb4fd82b49743abc2bb1989da0ec1b78a1dfbea14bbeb1d7dac02084bc7c194: Status 404 returned error can't find the container with id 3fb4fd82b49743abc2bb1989da0ec1b78a1dfbea14bbeb1d7dac02084bc7c194
Sep 15 11:38:53.235766 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:53.235730    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:38:53.235835 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:53.235797    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:38:53.235835 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:53.235828    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:38:53.725743 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:53.725705    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:54.031026 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:54.030938    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x" event={"ID":"b5375fb3-7899-4ad2-965d-61707dac6c93","Type":"ContainerStarted","Data":"40c46cda348656fc8b994e80083355630d082dd02eb1faad25293aada3c83cd2"}
Sep 15 11:38:54.031026 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:54.030984    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x" event={"ID":"b5375fb3-7899-4ad2-965d-61707dac6c93","Type":"ContainerStarted","Data":"3fb4fd82b49743abc2bb1989da0ec1b78a1dfbea14bbeb1d7dac02084bc7c194"}
Sep 15 11:38:54.057610 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:54.057555    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x" podStartSLOduration=2.057541951 podStartE2EDuration="2.057541951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:38:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:38:54.054907295 +0000 UTC m=+835.547684444" watchObservedRunningTime="2026-09-15 11:38:54.057541951 +0000 UTC m=+835.550319141"
Sep 15 11:38:54.676584 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:54.676527    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:54.949264 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:54.949173    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:54.954867 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:54.954842    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:55.036695 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:55.036659    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:55.037699 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:55.037678    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:38:55.645317 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:55.645285    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:55.815238 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:55.815205    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:56.783184 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:56.783136    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:57.729745 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:57.729714    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:57.803225 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:57.803182    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:58.262162 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.262124    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:58.352251 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.352216    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:58.453907 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.453867    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:58.462442 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.462415    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"]
Sep 15 11:38:58.465772 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.465757    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"
Sep 15 11:38:58.470518 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.470486    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-07a6c076-dockercfg-bm9j6\""
Sep 15 11:38:58.470668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.470497    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-07a6c076\""
Sep 15 11:38:58.492161 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.492134    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"]
Sep 15 11:38:58.618590 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.618524    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ac84b40-d0f2-4eb5-b9a4-36227f777cbb-plugins-config-volume\") pod \"payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk\" (UID: \"9ac84b40-d0f2-4eb5-b9a4-36227f777cbb\") " pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"
Sep 15 11:38:58.618781 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.618758    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwj8t\" (UniqueName: \"kubernetes.io/projected/9ac84b40-d0f2-4eb5-b9a4-36227f777cbb-kube-api-access-cwj8t\") pod \"payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk\" (UID: \"9ac84b40-d0f2-4eb5-b9a4-36227f777cbb\") " pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"
Sep 15 11:38:58.719490 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.719457    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwj8t\" (UniqueName: \"kubernetes.io/projected/9ac84b40-d0f2-4eb5-b9a4-36227f777cbb-kube-api-access-cwj8t\") pod \"payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk\" (UID: \"9ac84b40-d0f2-4eb5-b9a4-36227f777cbb\") " pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"
Sep 15 11:38:58.719490 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.719494    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ac84b40-d0f2-4eb5-b9a4-36227f777cbb-plugins-config-volume\") pod \"payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk\" (UID: \"9ac84b40-d0f2-4eb5-b9a4-36227f777cbb\") " pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"
Sep 15 11:38:58.720181 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.720158    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ac84b40-d0f2-4eb5-b9a4-36227f777cbb-plugins-config-volume\") pod \"payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk\" (UID: \"9ac84b40-d0f2-4eb5-b9a4-36227f777cbb\") " pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"
Sep 15 11:38:58.728093 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.728072    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwj8t\" (UniqueName: \"kubernetes.io/projected/9ac84b40-d0f2-4eb5-b9a4-36227f777cbb-kube-api-access-cwj8t\") pod \"payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk\" (UID: \"9ac84b40-d0f2-4eb5-b9a4-36227f777cbb\") " pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"
Sep 15 11:38:58.775684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.775660    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"
Sep 15 11:38:58.860270 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:58.860234    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:59.059046 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:59.059019    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"]
Sep 15 11:38:59.060307 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:38:59.060282    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac84b40_d0f2_4eb5_b9a4_36227f777cbb.slice/crio-666ce8dc6f26f5d1e9bdc589c027ab5f98a1d91287cecfc053ac0e3d0083c4bf WatchSource:0}: Error finding container 666ce8dc6f26f5d1e9bdc589c027ab5f98a1d91287cecfc053ac0e3d0083c4bf: Status 404 returned error can't find the container with id 666ce8dc6f26f5d1e9bdc589c027ab5f98a1d91287cecfc053ac0e3d0083c4bf
Sep 15 11:38:59.759500 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:59.759463    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:38:59.867037 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:38:59.866991    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:00.066728 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:00.066647    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk" event={"ID":"9ac84b40-d0f2-4eb5-b9a4-36227f777cbb","Type":"ContainerStarted","Data":"ef08b8b71d41d72bdf417da76963eaed0eeb211bcc17f07a16f241b39ecdbaf4"}
Sep 15 11:39:00.066728 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:00.066706    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk" event={"ID":"9ac84b40-d0f2-4eb5-b9a4-36227f777cbb","Type":"ContainerStarted","Data":"666ce8dc6f26f5d1e9bdc589c027ab5f98a1d91287cecfc053ac0e3d0083c4bf"}
Sep 15 11:39:00.066912 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:00.066755    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"
Sep 15 11:39:00.092969 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:00.092910    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk" podStartSLOduration=2.092890973 podStartE2EDuration="2.092890973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:38:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:39:00.092282842 +0000 UTC m=+841.585060001" watchObservedRunningTime="2026-09-15 11:39:00.092890973 +0000 UTC m=+841.585668122"
Sep 15 11:39:00.703136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:00.703098    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:01.579686 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:01.579651    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:02.394337 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:02.394303    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:11.073727 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:11.073698    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"
Sep 15 11:39:17.135131 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:17.135094    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:18.169335 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:18.169262    2560 generic.go:356] "Generic (PLEG): container finished" podID="b5375fb3-7899-4ad2-965d-61707dac6c93" containerID="40c46cda348656fc8b994e80083355630d082dd02eb1faad25293aada3c83cd2" exitCode=0
Sep 15 11:39:18.169335 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:18.169322    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x" event={"ID":"b5375fb3-7899-4ad2-965d-61707dac6c93","Type":"ContainerDied","Data":"40c46cda348656fc8b994e80083355630d082dd02eb1faad25293aada3c83cd2"}
Sep 15 11:39:18.169809 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:18.169748    2560 scope.go:117] "RemoveContainer" containerID="40c46cda348656fc8b994e80083355630d082dd02eb1faad25293aada3c83cd2"
Sep 15 11:39:18.170901 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:18.170869    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:39:18.170964 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:18.170939    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:39:18.171001 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:18.170970    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:39:19.177208 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:19.177162    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x" event={"ID":"b5375fb3-7899-4ad2-965d-61707dac6c93","Type":"ContainerStarted","Data":"6ed403611e45d6ae69860a87f3291b54f2e0d3d91d5ddb4ef2d95bc97c99efe0"}
Sep 15 11:39:19.948354 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:19.948319    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:39:19.948527 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:19.948469    2560 patch_prober.go:28] interesting pod/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 15 11:39:19.948603 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:19.948534    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x" podUID="b5375fb3-7899-4ad2-965d-61707dac6c93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 15 11:39:20.304374 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:20.304285    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:20.948672 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:20.948631    2560 patch_prober.go:28] interesting pod/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 15 11:39:20.948849 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:20.948702    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x" podUID="b5375fb3-7899-4ad2-965d-61707dac6c93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 15 11:39:21.327742 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:21.327648    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:21.948800 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:21.948762    2560 patch_prober.go:28] interesting pod/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 15 11:39:21.948973 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:21.948824    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x" podUID="b5375fb3-7899-4ad2-965d-61707dac6c93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 15 11:39:22.115472 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:22.115425    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:22.870473 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:22.870407    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:22.948920 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:22.948879    2560 patch_prober.go:28] interesting pod/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 15 11:39:22.949122 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:22.948957    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x" podUID="b5375fb3-7899-4ad2-965d-61707dac6c93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 15 11:39:23.729916 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:23.729881    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:23.949012 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:23.948976    2560 patch_prober.go:28] interesting pod/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 15 11:39:23.949388 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:23.949028    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x" podUID="b5375fb3-7899-4ad2-965d-61707dac6c93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 15 11:39:24.508848 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:24.508807    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:24.953456 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:24.953426    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:39:24.953932 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:24.953851    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:39:24.954484 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:24.954463    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:39:25.504402 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:25.504364    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:26.198346 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:26.198311    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:26.899195 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:26.899156    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:26.917265 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:26.917233    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"]
Sep 15 11:39:26.917474 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:26.917449    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk" podUID="9ac84b40-d0f2-4eb5-b9a4-36227f777cbb" containerName="payload-processing" containerID="cri-o://ef08b8b71d41d72bdf417da76963eaed0eeb211bcc17f07a16f241b39ecdbaf4" gracePeriod=30
Sep 15 11:39:26.997581 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:26.997533    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:27.099601 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.097983    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:27.169420 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.169392    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"
Sep 15 11:39:27.192812 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.192721    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:27.214780 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.212514    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwj8t\" (UniqueName: \"kubernetes.io/projected/9ac84b40-d0f2-4eb5-b9a4-36227f777cbb-kube-api-access-cwj8t\") pod \"9ac84b40-d0f2-4eb5-b9a4-36227f777cbb\" (UID: \"9ac84b40-d0f2-4eb5-b9a4-36227f777cbb\") "
Sep 15 11:39:27.214780 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.212661    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ac84b40-d0f2-4eb5-b9a4-36227f777cbb-plugins-config-volume\") pod \"9ac84b40-d0f2-4eb5-b9a4-36227f777cbb\" (UID: \"9ac84b40-d0f2-4eb5-b9a4-36227f777cbb\") "
Sep 15 11:39:27.214780 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.213778    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ac84b40-d0f2-4eb5-b9a4-36227f777cbb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9ac84b40-d0f2-4eb5-b9a4-36227f777cbb" (UID: "9ac84b40-d0f2-4eb5-b9a4-36227f777cbb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:39:27.217675 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.217630    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ac84b40-d0f2-4eb5-b9a4-36227f777cbb-kube-api-access-cwj8t" (OuterVolumeSpecName: "kube-api-access-cwj8t") pod "9ac84b40-d0f2-4eb5-b9a4-36227f777cbb" (UID: "9ac84b40-d0f2-4eb5-b9a4-36227f777cbb"). InnerVolumeSpecName "kube-api-access-cwj8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:39:27.232689 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.232651    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"]
Sep 15 11:39:27.240745 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.240715    2560 generic.go:356] "Generic (PLEG): container finished" podID="9ac84b40-d0f2-4eb5-b9a4-36227f777cbb" containerID="ef08b8b71d41d72bdf417da76963eaed0eeb211bcc17f07a16f241b39ecdbaf4" exitCode=0
Sep 15 11:39:27.240929 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.240910    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x" podUID="b5375fb3-7899-4ad2-965d-61707dac6c93" containerName="istio-proxy" containerID="cri-o://6ed403611e45d6ae69860a87f3291b54f2e0d3d91d5ddb4ef2d95bc97c99efe0" gracePeriod=30
Sep 15 11:39:27.241495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.241127    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"
Sep 15 11:39:27.241495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.241168    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk" event={"ID":"9ac84b40-d0f2-4eb5-b9a4-36227f777cbb","Type":"ContainerDied","Data":"ef08b8b71d41d72bdf417da76963eaed0eeb211bcc17f07a16f241b39ecdbaf4"}
Sep 15 11:39:27.241495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.241225    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk" event={"ID":"9ac84b40-d0f2-4eb5-b9a4-36227f777cbb","Type":"ContainerDied","Data":"666ce8dc6f26f5d1e9bdc589c027ab5f98a1d91287cecfc053ac0e3d0083c4bf"}
Sep 15 11:39:27.241495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.241257    2560 scope.go:117] "RemoveContainer" containerID="ef08b8b71d41d72bdf417da76963eaed0eeb211bcc17f07a16f241b39ecdbaf4"
Sep 15 11:39:27.251741 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.251723    2560 scope.go:117] "RemoveContainer" containerID="ef08b8b71d41d72bdf417da76963eaed0eeb211bcc17f07a16f241b39ecdbaf4"
Sep 15 11:39:27.252013 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:39:27.251991    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef08b8b71d41d72bdf417da76963eaed0eeb211bcc17f07a16f241b39ecdbaf4\": container with ID starting with ef08b8b71d41d72bdf417da76963eaed0eeb211bcc17f07a16f241b39ecdbaf4 not found: ID does not exist" containerID="ef08b8b71d41d72bdf417da76963eaed0eeb211bcc17f07a16f241b39ecdbaf4"
Sep 15 11:39:27.252073 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.252025    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef08b8b71d41d72bdf417da76963eaed0eeb211bcc17f07a16f241b39ecdbaf4"} err="failed to get container status \"ef08b8b71d41d72bdf417da76963eaed0eeb211bcc17f07a16f241b39ecdbaf4\": rpc error: code = NotFound desc = could not find container \"ef08b8b71d41d72bdf417da76963eaed0eeb211bcc17f07a16f241b39ecdbaf4\": container with ID starting with ef08b8b71d41d72bdf417da76963eaed0eeb211bcc17f07a16f241b39ecdbaf4 not found: ID does not exist"
Sep 15 11:39:27.265889 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.265845    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:27.273203 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.273180    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"]
Sep 15 11:39:27.279606 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.279585    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-07a6c076-7cc4676d74-vcmtk"]
Sep 15 11:39:27.314188 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.314163    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ac84b40-d0f2-4eb5-b9a4-36227f777cbb-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:39:27.314188 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.314187    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwj8t\" (UniqueName: \"kubernetes.io/projected/9ac84b40-d0f2-4eb5-b9a4-36227f777cbb-kube-api-access-cwj8t\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:39:27.694021 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.693992    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:27.793807 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.793770    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:27.958099 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.958015    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"]
Sep 15 11:39:27.958857 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.958833    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ac84b40-d0f2-4eb5-b9a4-36227f777cbb" containerName="payload-processing"
Sep 15 11:39:27.958857 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.958854    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ac84b40-d0f2-4eb5-b9a4-36227f777cbb" containerName="payload-processing"
Sep 15 11:39:27.959014 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.958943    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ac84b40-d0f2-4eb5-b9a4-36227f777cbb" containerName="payload-processing"
Sep 15 11:39:27.963492 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.963473    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:27.965587 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.965553    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9b1cbda7-openshift-default-dockercfg-znfct\""
Sep 15 11:39:27.974858 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.974837    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"]
Sep 15 11:39:27.995086 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:27.995049    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:28.021226 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.021203    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-credential-socket\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.021356 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.021248    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9whhh\" (UniqueName: \"kubernetes.io/projected/ba61d864-c491-4420-8b4b-7dd33b6947c5-kube-api-access-9whhh\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.021356 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.021314    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-workload-socket\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.021356 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.021348    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-envoy\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.021480 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.021386    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-workload-certs\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.021480 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.021404    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba61d864-c491-4420-8b4b-7dd33b6947c5-istiod-ca-cert\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.021480 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.021440    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-token\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.021617 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.021494    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-data\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.021617 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.021527    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-podinfo\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.122490 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.122450    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9whhh\" (UniqueName: \"kubernetes.io/projected/ba61d864-c491-4420-8b4b-7dd33b6947c5-kube-api-access-9whhh\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.122729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.122510    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-workload-socket\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.122729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.122537    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-envoy\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.122729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.122615    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-workload-certs\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.122729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.122644    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba61d864-c491-4420-8b4b-7dd33b6947c5-istiod-ca-cert\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.122729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.122704    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-token\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.123030 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.122763    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-data\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.123030 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.122821    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-podinfo\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.123030 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.122865    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-credential-socket\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.123030 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.122972    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-workload-socket\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.123030 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.123010    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-workload-certs\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.123310 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.123207    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-credential-socket\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.123310 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.123260    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-data\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.123409 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.123392    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba61d864-c491-4420-8b4b-7dd33b6947c5-istiod-ca-cert\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.124884 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.124860    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-envoy\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.125322 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.125302    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-podinfo\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.130434 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.130412    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-token\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.130626 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.130608    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9whhh\" (UniqueName: \"kubernetes.io/projected/ba61d864-c491-4420-8b4b-7dd33b6947c5-kube-api-access-9whhh\") pod \"e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") " pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.163672 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.163637    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:28.274313 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.274227    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:28.434689 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.434650    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:28.555126 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.555101    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"]
Sep 15 11:39:28.557209 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:39:28.557184    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba61d864_c491_4420_8b4b_7dd33b6947c5.slice/crio-264042f0bd6346df38ef1604e3117612de15c32e2e15a457bced84d0317b6225 WatchSource:0}: Error finding container 264042f0bd6346df38ef1604e3117612de15c32e2e15a457bced84d0317b6225: Status 404 returned error can't find the container with id 264042f0bd6346df38ef1604e3117612de15c32e2e15a457bced84d0317b6225
Sep 15 11:39:28.559861 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.559825    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:39:28.559968 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.559902    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:39:28.559968 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:28.559946    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:39:29.039336 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:29.039010    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ac84b40-d0f2-4eb5-b9a4-36227f777cbb" path="/var/lib/kubelet/pods/9ac84b40-d0f2-4eb5-b9a4-36227f777cbb/volumes"
Sep 15 11:39:29.254266 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:29.254217    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g" event={"ID":"ba61d864-c491-4420-8b4b-7dd33b6947c5","Type":"ContainerStarted","Data":"d61025ff75996ea396aad62a6e9e63c246660fb889f73ed721f24e511c7b3ec9"}
Sep 15 11:39:29.254465 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:29.254290    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g" event={"ID":"ba61d864-c491-4420-8b4b-7dd33b6947c5","Type":"ContainerStarted","Data":"264042f0bd6346df38ef1604e3117612de15c32e2e15a457bced84d0317b6225"}
Sep 15 11:39:29.274512 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:29.274484    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:29.279769 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:29.279724    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g" podStartSLOduration=2.2797092230000002 podStartE2EDuration="2.279709223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:39:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:39:29.276686563 +0000 UTC m=+870.769463711" watchObservedRunningTime="2026-09-15 11:39:29.279709223 +0000 UTC m=+870.772486371"
Sep 15 11:39:29.280050 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:29.280032    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:29.292248 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:29.292171    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:30.081785 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:30.081753    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:30.260299 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:30.260272    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:30.261372 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:30.261351    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:30.628194 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:30.628154    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:30.715859 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:30.715817    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:30.922805 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:30.922712    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:32.015234 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.015195    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:32.505546 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.505521    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:39:32.568858 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.568825    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-podinfo\") pod \"b5375fb3-7899-4ad2-965d-61707dac6c93\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") "
Sep 15 11:39:32.569045 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.568870    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-token\") pod \"b5375fb3-7899-4ad2-965d-61707dac6c93\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") "
Sep 15 11:39:32.569045 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.568891    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-credential-socket\") pod \"b5375fb3-7899-4ad2-965d-61707dac6c93\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") "
Sep 15 11:39:32.569045 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.568919    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5375fb3-7899-4ad2-965d-61707dac6c93-istiod-ca-cert\") pod \"b5375fb3-7899-4ad2-965d-61707dac6c93\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") "
Sep 15 11:39:32.569045 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.568948    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-envoy\") pod \"b5375fb3-7899-4ad2-965d-61707dac6c93\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") "
Sep 15 11:39:32.569045 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.568983    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-workload-certs\") pod \"b5375fb3-7899-4ad2-965d-61707dac6c93\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") "
Sep 15 11:39:32.569045 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.569043    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-data\") pod \"b5375fb3-7899-4ad2-965d-61707dac6c93\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") "
Sep 15 11:39:32.569365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.569106    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwjcz\" (UniqueName: \"kubernetes.io/projected/b5375fb3-7899-4ad2-965d-61707dac6c93-kube-api-access-zwjcz\") pod \"b5375fb3-7899-4ad2-965d-61707dac6c93\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") "
Sep 15 11:39:32.569365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.569145    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-workload-socket\") pod \"b5375fb3-7899-4ad2-965d-61707dac6c93\" (UID: \"b5375fb3-7899-4ad2-965d-61707dac6c93\") "
Sep 15 11:39:32.569365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.569221    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b5375fb3-7899-4ad2-965d-61707dac6c93" (UID: "b5375fb3-7899-4ad2-965d-61707dac6c93"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:39:32.569365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.569321    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b5375fb3-7899-4ad2-965d-61707dac6c93" (UID: "b5375fb3-7899-4ad2-965d-61707dac6c93"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:39:32.569601 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.569443    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-data" (OuterVolumeSpecName: "istio-data") pod "b5375fb3-7899-4ad2-965d-61707dac6c93" (UID: "b5375fb3-7899-4ad2-965d-61707dac6c93"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:39:32.569601 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.569459    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5375fb3-7899-4ad2-965d-61707dac6c93-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b5375fb3-7899-4ad2-965d-61707dac6c93" (UID: "b5375fb3-7899-4ad2-965d-61707dac6c93"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:39:32.569601 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.569580    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b5375fb3-7899-4ad2-965d-61707dac6c93" (UID: "b5375fb3-7899-4ad2-965d-61707dac6c93"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:39:32.569601 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.569586    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:39:32.569847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.569623    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5375fb3-7899-4ad2-965d-61707dac6c93-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:39:32.569847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.569641    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:39:32.569847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.569653    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:39:32.571279 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.571257    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-token" (OuterVolumeSpecName: "istio-token") pod "b5375fb3-7899-4ad2-965d-61707dac6c93" (UID: "b5375fb3-7899-4ad2-965d-61707dac6c93"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:39:32.571391 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.571336    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b5375fb3-7899-4ad2-965d-61707dac6c93" (UID: "b5375fb3-7899-4ad2-965d-61707dac6c93"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:39:32.571767 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.571741    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5375fb3-7899-4ad2-965d-61707dac6c93-kube-api-access-zwjcz" (OuterVolumeSpecName: "kube-api-access-zwjcz") pod "b5375fb3-7899-4ad2-965d-61707dac6c93" (UID: "b5375fb3-7899-4ad2-965d-61707dac6c93"). InnerVolumeSpecName "kube-api-access-zwjcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:39:32.571767 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.571755    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b5375fb3-7899-4ad2-965d-61707dac6c93" (UID: "b5375fb3-7899-4ad2-965d-61707dac6c93"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:39:32.670678 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.670589    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:39:32.670678 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.670628    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:39:32.670678 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.670643    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:39:32.670678 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.670655    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5375fb3-7899-4ad2-965d-61707dac6c93-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:39:32.670678 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.670669    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwjcz\" (UniqueName: \"kubernetes.io/projected/b5375fb3-7899-4ad2-965d-61707dac6c93-kube-api-access-zwjcz\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:39:32.810455 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.810421    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:32.899306 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:32.899263    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:33.199551 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.199511    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:33.280119 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.280079    2560 generic.go:356] "Generic (PLEG): container finished" podID="b5375fb3-7899-4ad2-965d-61707dac6c93" containerID="6ed403611e45d6ae69860a87f3291b54f2e0d3d91d5ddb4ef2d95bc97c99efe0" exitCode=0
Sep 15 11:39:33.280262 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.280142    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"
Sep 15 11:39:33.280262 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.280153    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x" event={"ID":"b5375fb3-7899-4ad2-965d-61707dac6c93","Type":"ContainerDied","Data":"6ed403611e45d6ae69860a87f3291b54f2e0d3d91d5ddb4ef2d95bc97c99efe0"}
Sep 15 11:39:33.280262 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.280227    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x" event={"ID":"b5375fb3-7899-4ad2-965d-61707dac6c93","Type":"ContainerDied","Data":"3fb4fd82b49743abc2bb1989da0ec1b78a1dfbea14bbeb1d7dac02084bc7c194"}
Sep 15 11:39:33.280262 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.280249    2560 scope.go:117] "RemoveContainer" containerID="6ed403611e45d6ae69860a87f3291b54f2e0d3d91d5ddb4ef2d95bc97c99efe0"
Sep 15 11:39:33.290061 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.290039    2560 scope.go:117] "RemoveContainer" containerID="40c46cda348656fc8b994e80083355630d082dd02eb1faad25293aada3c83cd2"
Sep 15 11:39:33.298827 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.298811    2560 scope.go:117] "RemoveContainer" containerID="6ed403611e45d6ae69860a87f3291b54f2e0d3d91d5ddb4ef2d95bc97c99efe0"
Sep 15 11:39:33.299097 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:39:33.299072    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ed403611e45d6ae69860a87f3291b54f2e0d3d91d5ddb4ef2d95bc97c99efe0\": container with ID starting with 6ed403611e45d6ae69860a87f3291b54f2e0d3d91d5ddb4ef2d95bc97c99efe0 not found: ID does not exist" containerID="6ed403611e45d6ae69860a87f3291b54f2e0d3d91d5ddb4ef2d95bc97c99efe0"
Sep 15 11:39:33.299162 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.299100    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ed403611e45d6ae69860a87f3291b54f2e0d3d91d5ddb4ef2d95bc97c99efe0"} err="failed to get container status \"6ed403611e45d6ae69860a87f3291b54f2e0d3d91d5ddb4ef2d95bc97c99efe0\": rpc error: code = NotFound desc = could not find container \"6ed403611e45d6ae69860a87f3291b54f2e0d3d91d5ddb4ef2d95bc97c99efe0\": container with ID starting with 6ed403611e45d6ae69860a87f3291b54f2e0d3d91d5ddb4ef2d95bc97c99efe0 not found: ID does not exist"
Sep 15 11:39:33.299162 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.299120    2560 scope.go:117] "RemoveContainer" containerID="40c46cda348656fc8b994e80083355630d082dd02eb1faad25293aada3c83cd2"
Sep 15 11:39:33.299342 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:39:33.299326    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40c46cda348656fc8b994e80083355630d082dd02eb1faad25293aada3c83cd2\": container with ID starting with 40c46cda348656fc8b994e80083355630d082dd02eb1faad25293aada3c83cd2 not found: ID does not exist" containerID="40c46cda348656fc8b994e80083355630d082dd02eb1faad25293aada3c83cd2"
Sep 15 11:39:33.299389 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.299345    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40c46cda348656fc8b994e80083355630d082dd02eb1faad25293aada3c83cd2"} err="failed to get container status \"40c46cda348656fc8b994e80083355630d082dd02eb1faad25293aada3c83cd2\": rpc error: code = NotFound desc = could not find container \"40c46cda348656fc8b994e80083355630d082dd02eb1faad25293aada3c83cd2\": container with ID starting with 40c46cda348656fc8b994e80083355630d082dd02eb1faad25293aada3c83cd2 not found: ID does not exist"
Sep 15 11:39:33.305555 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.305530    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"]
Sep 15 11:39:33.309272 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.309251    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-07a6c076-gw-openshift-default-5596775f6-zjs9x"]
Sep 15 11:39:33.538806 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.538766    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:33.654015 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.653980    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:33.854765 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.854728    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"]
Sep 15 11:39:33.855422 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.855400    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5375fb3-7899-4ad2-965d-61707dac6c93" containerName="istio-proxy"
Sep 15 11:39:33.855537 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.855424    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5375fb3-7899-4ad2-965d-61707dac6c93" containerName="istio-proxy"
Sep 15 11:39:33.855633 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.855580    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5375fb3-7899-4ad2-965d-61707dac6c93" containerName="istio-proxy"
Sep 15 11:39:33.855633 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.855597    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5375fb3-7899-4ad2-965d-61707dac6c93" containerName="istio-proxy"
Sep 15 11:39:33.860706 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.860684    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"
Sep 15 11:39:33.863668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.863451    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-9b1cbda7-dockercfg-prg2s\""
Sep 15 11:39:33.863668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.863646    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-9b1cbda7\""
Sep 15 11:39:33.936439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.936355    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"]
Sep 15 11:39:33.986470 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.986438    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a94a7a4-d226-425c-b4f0-73e5af34da8a-plugins-config-volume\") pod \"payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52\" (UID: \"5a94a7a4-d226-425c-b4f0-73e5af34da8a\") " pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"
Sep 15 11:39:33.986684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:33.986497    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xbrw\" (UniqueName: \"kubernetes.io/projected/5a94a7a4-d226-425c-b4f0-73e5af34da8a-kube-api-access-9xbrw\") pod \"payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52\" (UID: \"5a94a7a4-d226-425c-b4f0-73e5af34da8a\") " pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"
Sep 15 11:39:34.087949 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:34.087907    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a94a7a4-d226-425c-b4f0-73e5af34da8a-plugins-config-volume\") pod \"payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52\" (UID: \"5a94a7a4-d226-425c-b4f0-73e5af34da8a\") " pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"
Sep 15 11:39:34.088146 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:34.087987    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xbrw\" (UniqueName: \"kubernetes.io/projected/5a94a7a4-d226-425c-b4f0-73e5af34da8a-kube-api-access-9xbrw\") pod \"payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52\" (UID: \"5a94a7a4-d226-425c-b4f0-73e5af34da8a\") " pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"
Sep 15 11:39:34.088997 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:34.088968    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a94a7a4-d226-425c-b4f0-73e5af34da8a-plugins-config-volume\") pod \"payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52\" (UID: \"5a94a7a4-d226-425c-b4f0-73e5af34da8a\") " pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"
Sep 15 11:39:34.098234 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:34.098206    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xbrw\" (UniqueName: \"kubernetes.io/projected/5a94a7a4-d226-425c-b4f0-73e5af34da8a-kube-api-access-9xbrw\") pod \"payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52\" (UID: \"5a94a7a4-d226-425c-b4f0-73e5af34da8a\") " pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"
Sep 15 11:39:34.176782 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:34.176758    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"
Sep 15 11:39:34.282246 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:34.282207    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:34.462850 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:34.462822    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"]
Sep 15 11:39:34.464125 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:39:34.464100    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a94a7a4_d226_425c_b4f0_73e5af34da8a.slice/crio-9e5508180bacb85b53dc8641e4d0db6837a7356fc86a47ebbf01f5350108a32a WatchSource:0}: Error finding container 9e5508180bacb85b53dc8641e4d0db6837a7356fc86a47ebbf01f5350108a32a: Status 404 returned error can't find the container with id 9e5508180bacb85b53dc8641e4d0db6837a7356fc86a47ebbf01f5350108a32a
Sep 15 11:39:34.786993 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:34.786816    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:35.035542 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:35.035510    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5375fb3-7899-4ad2-965d-61707dac6c93" path="/var/lib/kubelet/pods/b5375fb3-7899-4ad2-965d-61707dac6c93/volumes"
Sep 15 11:39:35.238412 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:35.238375    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:35.300303 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:35.300246    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52" event={"ID":"5a94a7a4-d226-425c-b4f0-73e5af34da8a","Type":"ContainerStarted","Data":"a2e72f53540ae66cd47c279df63921fa0b7bab0515a8d556eacc9e721bec2e74"}
Sep 15 11:39:35.300874 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:35.300331    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52" event={"ID":"5a94a7a4-d226-425c-b4f0-73e5af34da8a","Type":"ContainerStarted","Data":"9e5508180bacb85b53dc8641e4d0db6837a7356fc86a47ebbf01f5350108a32a"}
Sep 15 11:39:35.300874 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:35.300678    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"
Sep 15 11:39:35.324143 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:35.324092    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52" podStartSLOduration=2.324076966 podStartE2EDuration="2.324076966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:39:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:39:35.323408197 +0000 UTC m=+876.816185347" watchObservedRunningTime="2026-09-15 11:39:35.324076966 +0000 UTC m=+876.816854146"
Sep 15 11:39:35.982759 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:35.982723    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:36.709999 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:36.709964    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:43.569946 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:43.569905    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:46.309296 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:46.309264    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"
Sep 15 11:39:54.146064 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:54.146022    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:55.575742 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:55.575703    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:56.264078 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.264035    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:56.338105 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.338031    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v"]
Sep 15 11:39:56.338798 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.338772    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5375fb3-7899-4ad2-965d-61707dac6c93" containerName="istio-proxy"
Sep 15 11:39:56.338798 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.338799    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5375fb3-7899-4ad2-965d-61707dac6c93" containerName="istio-proxy"
Sep 15 11:39:56.342223 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.342205    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v"
Sep 15 11:39:56.344239 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.344222    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbxjk\""
Sep 15 11:39:56.348413 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.348385    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v"]
Sep 15 11:39:56.413454 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.413407    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c791c4a2-5fa0-4e29-8629-0b5f577f284f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v\" (UID: \"c791c4a2-5fa0-4e29-8629-0b5f577f284f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v"
Sep 15 11:39:56.419408 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.419379    2560 generic.go:356] "Generic (PLEG): container finished" podID="ba61d864-c491-4420-8b4b-7dd33b6947c5" containerID="d61025ff75996ea396aad62a6e9e63c246660fb889f73ed721f24e511c7b3ec9" exitCode=0
Sep 15 11:39:56.419540 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.419453    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g" event={"ID":"ba61d864-c491-4420-8b4b-7dd33b6947c5","Type":"ContainerDied","Data":"d61025ff75996ea396aad62a6e9e63c246660fb889f73ed721f24e511c7b3ec9"}
Sep 15 11:39:56.419952 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.419932    2560 scope.go:117] "RemoveContainer" containerID="d61025ff75996ea396aad62a6e9e63c246660fb889f73ed721f24e511c7b3ec9"
Sep 15 11:39:56.421304 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.421273    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:39:56.421376 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.421351    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:39:56.421414 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.421384    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:39:56.514644 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.514544    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c791c4a2-5fa0-4e29-8629-0b5f577f284f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v\" (UID: \"c791c4a2-5fa0-4e29-8629-0b5f577f284f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v"
Sep 15 11:39:56.515367 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.515346    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c791c4a2-5fa0-4e29-8629-0b5f577f284f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v\" (UID: \"c791c4a2-5fa0-4e29-8629-0b5f577f284f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v"
Sep 15 11:39:56.655232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.655197    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v"
Sep 15 11:39:56.935540 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:56.935509    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v"]
Sep 15 11:39:56.937517 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:39:56.937491    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc791c4a2_5fa0_4e29_8629_0b5f577f284f.slice/crio-fc0f33d06e9bf15ffa76934befb196d70455d65c6a42273cd4527204308fa842 WatchSource:0}: Error finding container fc0f33d06e9bf15ffa76934befb196d70455d65c6a42273cd4527204308fa842: Status 404 returned error can't find the container with id fc0f33d06e9bf15ffa76934befb196d70455d65c6a42273cd4527204308fa842
Sep 15 11:39:57.078478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:57.078448    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:57.426800 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:57.426758    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v" event={"ID":"c791c4a2-5fa0-4e29-8629-0b5f577f284f","Type":"ContainerStarted","Data":"fc0f33d06e9bf15ffa76934befb196d70455d65c6a42273cd4527204308fa842"}
Sep 15 11:39:57.428726 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:57.428696    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g" event={"ID":"ba61d864-c491-4420-8b4b-7dd33b6947c5","Type":"ContainerStarted","Data":"d907324b13d3fc4afa027dc99eb429c5db1310ab58b018fb25e24a1249ec0411"}
Sep 15 11:39:57.905586 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:57.901318    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:58.274786 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:58.274689    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:39:58.276005 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:58.275958    2560 patch_prober.go:28] interesting pod/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 15 11:39:58.276163 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:58.276017    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g" podUID="ba61d864-c491-4420-8b4b-7dd33b6947c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 15 11:39:58.885799 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:58.885760    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:39:59.274886 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:59.274807    2560 patch_prober.go:28] interesting pod/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 15 11:39:59.274886 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:59.274872    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g" podUID="ba61d864-c491-4420-8b4b-7dd33b6947c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 15 11:39:59.561058 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:59.561030    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:39:59.562199 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:59.562174    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:39:59.619849 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:39:59.619820    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:00.261245 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:00.261211    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:40:00.274955 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:00.274926    2560 patch_prober.go:28] interesting pod/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 15 11:40:00.275393 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:00.274981    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g" podUID="ba61d864-c491-4420-8b4b-7dd33b6947c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 15 11:40:00.382843 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:00.382786    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:00.452012 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:00.451160    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v" event={"ID":"c791c4a2-5fa0-4e29-8629-0b5f577f284f","Type":"ContainerStarted","Data":"3e871907d890f4bb369b09f750d63b9d6e52c850a6e4383a4380af2014ea1f77"}
Sep 15 11:40:00.472321 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:00.472265    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v" podStartSLOduration=1.79162819 podStartE2EDuration="4.472249836s" totalImagesPullingTime="2.680621646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:39:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:40:00.469915593 +0000 UTC m=+901.962692739" watchObservedRunningTime="2026-09-15 11:40:00.472249836 +0000 UTC m=+901.965026983"
Sep 15 11:40:01.275093 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:01.275061    2560 patch_prober.go:28] interesting pod/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 15 11:40:01.275476 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:01.275112    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g" podUID="ba61d864-c491-4420-8b4b-7dd33b6947c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 15 11:40:01.325399 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:01.325359    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:01.457485 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:01.457443    2560 generic.go:356] "Generic (PLEG): container finished" podID="c791c4a2-5fa0-4e29-8629-0b5f577f284f" containerID="3e871907d890f4bb369b09f750d63b9d6e52c850a6e4383a4380af2014ea1f77" exitCode=0
Sep 15 11:40:01.457695 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:01.457510    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v" event={"ID":"c791c4a2-5fa0-4e29-8629-0b5f577f284f","Type":"ContainerDied","Data":"3e871907d890f4bb369b09f750d63b9d6e52c850a6e4383a4380af2014ea1f77"}
Sep 15 11:40:02.275031 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:02.274989    2560 patch_prober.go:28] interesting pod/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 15 11:40:02.275197 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:02.275045    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g" podUID="ba61d864-c491-4420-8b4b-7dd33b6947c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 15 11:40:02.328006 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:02.327943    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:02.612255 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:02.612224    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v"
Sep 15 11:40:02.676822 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:02.676789    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c791c4a2-5fa0-4e29-8629-0b5f577f284f-maas-api-service-ca\") pod \"c791c4a2-5fa0-4e29-8629-0b5f577f284f\" (UID: \"c791c4a2-5fa0-4e29-8629-0b5f577f284f\") "
Sep 15 11:40:02.677180 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:02.677154    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c791c4a2-5fa0-4e29-8629-0b5f577f284f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c791c4a2-5fa0-4e29-8629-0b5f577f284f" (UID: "c791c4a2-5fa0-4e29-8629-0b5f577f284f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:40:02.778306 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:02.778272    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c791c4a2-5fa0-4e29-8629-0b5f577f284f-maas-api-service-ca\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:40:03.003494 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:03.003461    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:03.278683 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:03.278608    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:40:03.279649 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:03.279628    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:40:03.472218 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:03.472186    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v"
Sep 15 11:40:03.472414 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:03.472217    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v" event={"ID":"c791c4a2-5fa0-4e29-8629-0b5f577f284f","Type":"ContainerDied","Data":"fc0f33d06e9bf15ffa76934befb196d70455d65c6a42273cd4527204308fa842"}
Sep 15 11:40:03.472414 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:03.472275    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc0f33d06e9bf15ffa76934befb196d70455d65c6a42273cd4527204308fa842"
Sep 15 11:40:03.926646 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:03.926612    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:04.540839 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:04.540809    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:05.302197 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:05.302163    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:06.279587 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:06.279532    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:06.948730 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:06.948698    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:07.741806 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:07.741773    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:12.074725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:12.074681    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:12.704602 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:12.704571    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"]
Sep 15 11:40:12.704812 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:12.704786    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d" podUID="cd77a9cd-6675-4093-92b0-f9fef8723a2c" containerName="payload-pre-processing" containerID="cri-o://59fc8a4f998ae8d8cb2f167db1f3d0e69b45631833a2f226ca33f2250c38962b" gracePeriod=30
Sep 15 11:40:12.894386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:12.894345    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"]
Sep 15 11:40:12.894913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:12.894852    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52" podUID="5a94a7a4-d226-425c-b4f0-73e5af34da8a" containerName="payload-processing" containerID="cri-o://a2e72f53540ae66cd47c279df63921fa0b7bab0515a8d556eacc9e721bec2e74" gracePeriod=30
Sep 15 11:40:13.040329 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.040196    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"
Sep 15 11:40:13.171513 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.171488    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"
Sep 15 11:40:13.180335 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.180315    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hb598\" (UniqueName: \"kubernetes.io/projected/cd77a9cd-6675-4093-92b0-f9fef8723a2c-kube-api-access-hb598\") pod \"cd77a9cd-6675-4093-92b0-f9fef8723a2c\" (UID: \"cd77a9cd-6675-4093-92b0-f9fef8723a2c\") "
Sep 15 11:40:13.180451 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.180440    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd77a9cd-6675-4093-92b0-f9fef8723a2c-plugins-config-volume\") pod \"cd77a9cd-6675-4093-92b0-f9fef8723a2c\" (UID: \"cd77a9cd-6675-4093-92b0-f9fef8723a2c\") "
Sep 15 11:40:13.180802 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.180780    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd77a9cd-6675-4093-92b0-f9fef8723a2c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cd77a9cd-6675-4093-92b0-f9fef8723a2c" (UID: "cd77a9cd-6675-4093-92b0-f9fef8723a2c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:40:13.182313 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.182289    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd77a9cd-6675-4093-92b0-f9fef8723a2c-kube-api-access-hb598" (OuterVolumeSpecName: "kube-api-access-hb598") pod "cd77a9cd-6675-4093-92b0-f9fef8723a2c" (UID: "cd77a9cd-6675-4093-92b0-f9fef8723a2c"). InnerVolumeSpecName "kube-api-access-hb598". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:40:13.281281 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.281188    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a94a7a4-d226-425c-b4f0-73e5af34da8a-plugins-config-volume\") pod \"5a94a7a4-d226-425c-b4f0-73e5af34da8a\" (UID: \"5a94a7a4-d226-425c-b4f0-73e5af34da8a\") "
Sep 15 11:40:13.281761 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.281734    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xbrw\" (UniqueName: \"kubernetes.io/projected/5a94a7a4-d226-425c-b4f0-73e5af34da8a-kube-api-access-9xbrw\") pod \"5a94a7a4-d226-425c-b4f0-73e5af34da8a\" (UID: \"5a94a7a4-d226-425c-b4f0-73e5af34da8a\") "
Sep 15 11:40:13.281902 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.281638    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a94a7a4-d226-425c-b4f0-73e5af34da8a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5a94a7a4-d226-425c-b4f0-73e5af34da8a" (UID: "5a94a7a4-d226-425c-b4f0-73e5af34da8a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:40:13.282112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.282092    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hb598\" (UniqueName: \"kubernetes.io/projected/cd77a9cd-6675-4093-92b0-f9fef8723a2c-kube-api-access-hb598\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:40:13.282220 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.282113    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a94a7a4-d226-425c-b4f0-73e5af34da8a-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:40:13.282220 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.282122    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd77a9cd-6675-4093-92b0-f9fef8723a2c-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:40:13.284379 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.284348    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a94a7a4-d226-425c-b4f0-73e5af34da8a-kube-api-access-9xbrw" (OuterVolumeSpecName: "kube-api-access-9xbrw") pod "5a94a7a4-d226-425c-b4f0-73e5af34da8a" (UID: "5a94a7a4-d226-425c-b4f0-73e5af34da8a"). InnerVolumeSpecName "kube-api-access-9xbrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:40:13.325337 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.325300    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v"]
Sep 15 11:40:13.330164 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.330135    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9b1cbda7-4vf6v"]
Sep 15 11:40:13.383412 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.383372    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xbrw\" (UniqueName: \"kubernetes.io/projected/5a94a7a4-d226-425c-b4f0-73e5af34da8a-kube-api-access-9xbrw\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:40:13.536349 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.536245    2560 generic.go:356] "Generic (PLEG): container finished" podID="5a94a7a4-d226-425c-b4f0-73e5af34da8a" containerID="a2e72f53540ae66cd47c279df63921fa0b7bab0515a8d556eacc9e721bec2e74" exitCode=0
Sep 15 11:40:13.536349 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.536306    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"
Sep 15 11:40:13.536611 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.536372    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52" event={"ID":"5a94a7a4-d226-425c-b4f0-73e5af34da8a","Type":"ContainerDied","Data":"a2e72f53540ae66cd47c279df63921fa0b7bab0515a8d556eacc9e721bec2e74"}
Sep 15 11:40:13.536611 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.536443    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52" event={"ID":"5a94a7a4-d226-425c-b4f0-73e5af34da8a","Type":"ContainerDied","Data":"9e5508180bacb85b53dc8641e4d0db6837a7356fc86a47ebbf01f5350108a32a"}
Sep 15 11:40:13.536611 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.536469    2560 scope.go:117] "RemoveContainer" containerID="a2e72f53540ae66cd47c279df63921fa0b7bab0515a8d556eacc9e721bec2e74"
Sep 15 11:40:13.538750 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.538720    2560 generic.go:356] "Generic (PLEG): container finished" podID="cd77a9cd-6675-4093-92b0-f9fef8723a2c" containerID="59fc8a4f998ae8d8cb2f167db1f3d0e69b45631833a2f226ca33f2250c38962b" exitCode=0
Sep 15 11:40:13.538916 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.538888    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d" event={"ID":"cd77a9cd-6675-4093-92b0-f9fef8723a2c","Type":"ContainerDied","Data":"59fc8a4f998ae8d8cb2f167db1f3d0e69b45631833a2f226ca33f2250c38962b"}
Sep 15 11:40:13.539018 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.538952    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d" event={"ID":"cd77a9cd-6675-4093-92b0-f9fef8723a2c","Type":"ContainerDied","Data":"c92fcf0cfa81802a8cc0bc153e03320132611f502e6a239945d9dd05265ae2db"}
Sep 15 11:40:13.539129 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.539108    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"
Sep 15 11:40:13.558192 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.558169    2560 scope.go:117] "RemoveContainer" containerID="a2e72f53540ae66cd47c279df63921fa0b7bab0515a8d556eacc9e721bec2e74"
Sep 15 11:40:13.558607 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:40:13.558581    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2e72f53540ae66cd47c279df63921fa0b7bab0515a8d556eacc9e721bec2e74\": container with ID starting with a2e72f53540ae66cd47c279df63921fa0b7bab0515a8d556eacc9e721bec2e74 not found: ID does not exist" containerID="a2e72f53540ae66cd47c279df63921fa0b7bab0515a8d556eacc9e721bec2e74"
Sep 15 11:40:13.558742 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.558618    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2e72f53540ae66cd47c279df63921fa0b7bab0515a8d556eacc9e721bec2e74"} err="failed to get container status \"a2e72f53540ae66cd47c279df63921fa0b7bab0515a8d556eacc9e721bec2e74\": rpc error: code = NotFound desc = could not find container \"a2e72f53540ae66cd47c279df63921fa0b7bab0515a8d556eacc9e721bec2e74\": container with ID starting with a2e72f53540ae66cd47c279df63921fa0b7bab0515a8d556eacc9e721bec2e74 not found: ID does not exist"
Sep 15 11:40:13.558742 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.558645    2560 scope.go:117] "RemoveContainer" containerID="59fc8a4f998ae8d8cb2f167db1f3d0e69b45631833a2f226ca33f2250c38962b"
Sep 15 11:40:13.567090 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.567067    2560 scope.go:117] "RemoveContainer" containerID="59fc8a4f998ae8d8cb2f167db1f3d0e69b45631833a2f226ca33f2250c38962b"
Sep 15 11:40:13.567369 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:40:13.567346    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59fc8a4f998ae8d8cb2f167db1f3d0e69b45631833a2f226ca33f2250c38962b\": container with ID starting with 59fc8a4f998ae8d8cb2f167db1f3d0e69b45631833a2f226ca33f2250c38962b not found: ID does not exist" containerID="59fc8a4f998ae8d8cb2f167db1f3d0e69b45631833a2f226ca33f2250c38962b"
Sep 15 11:40:13.567467 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.567376    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59fc8a4f998ae8d8cb2f167db1f3d0e69b45631833a2f226ca33f2250c38962b"} err="failed to get container status \"59fc8a4f998ae8d8cb2f167db1f3d0e69b45631833a2f226ca33f2250c38962b\": rpc error: code = NotFound desc = could not find container \"59fc8a4f998ae8d8cb2f167db1f3d0e69b45631833a2f226ca33f2250c38962b\": container with ID starting with 59fc8a4f998ae8d8cb2f167db1f3d0e69b45631833a2f226ca33f2250c38962b not found: ID does not exist"
Sep 15 11:40:13.575516 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.575491    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"]
Sep 15 11:40:13.579431 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.579405    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9b1cbda7-74779bbb77-vpb52"]
Sep 15 11:40:13.594156 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.594127    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"]
Sep 15 11:40:13.599133 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:13.599109    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-4d6dbb-648b4c496b-vdh4d"]
Sep 15 11:40:14.125431 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:14.125394    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:14.239153 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:14.239116    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:14.426909 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:14.426827    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"]
Sep 15 11:40:14.427116 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:14.427083    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g" podUID="ba61d864-c491-4420-8b4b-7dd33b6947c5" containerName="istio-proxy" containerID="cri-o://d907324b13d3fc4afa027dc99eb429c5db1310ab58b018fb25e24a1249ec0411" gracePeriod=30
Sep 15 11:40:14.450417 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:14.450386    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:14.884550 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:14.884507    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:15.034525 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.034483    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a94a7a4-d226-425c-b4f0-73e5af34da8a" path="/var/lib/kubelet/pods/5a94a7a4-d226-425c-b4f0-73e5af34da8a/volumes"
Sep 15 11:40:15.034981 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.034956    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c791c4a2-5fa0-4e29-8629-0b5f577f284f" path="/var/lib/kubelet/pods/c791c4a2-5fa0-4e29-8629-0b5f577f284f/volumes"
Sep 15 11:40:15.035359 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.035340    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd77a9cd-6675-4093-92b0-f9fef8723a2c" path="/var/lib/kubelet/pods/cd77a9cd-6675-4093-92b0-f9fef8723a2c/volumes"
Sep 15 11:40:15.164869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.164780    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"]
Sep 15 11:40:15.165489 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.165467    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c791c4a2-5fa0-4e29-8629-0b5f577f284f" containerName="revoke-keys"
Sep 15 11:40:15.165542 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.165494    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c791c4a2-5fa0-4e29-8629-0b5f577f284f" containerName="revoke-keys"
Sep 15 11:40:15.165542 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.165515    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a94a7a4-d226-425c-b4f0-73e5af34da8a" containerName="payload-processing"
Sep 15 11:40:15.165542 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.165527    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a94a7a4-d226-425c-b4f0-73e5af34da8a" containerName="payload-processing"
Sep 15 11:40:15.165700 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.165540    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd77a9cd-6675-4093-92b0-f9fef8723a2c" containerName="payload-pre-processing"
Sep 15 11:40:15.165700 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.165551    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd77a9cd-6675-4093-92b0-f9fef8723a2c" containerName="payload-pre-processing"
Sep 15 11:40:15.165780 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.165709    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a94a7a4-d226-425c-b4f0-73e5af34da8a" containerName="payload-processing"
Sep 15 11:40:15.165780 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.165727    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c791c4a2-5fa0-4e29-8629-0b5f577f284f" containerName="revoke-keys"
Sep 15 11:40:15.165780 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.165738    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd77a9cd-6675-4093-92b0-f9fef8723a2c" containerName="payload-pre-processing"
Sep 15 11:40:15.168051 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.168023    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:15.171262 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.171235    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.179360 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.179342    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-6f21996a-openshift-default-dockercfg-fqqp5\""
Sep 15 11:40:15.186328 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.186301    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"]
Sep 15 11:40:15.259834 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.259800    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:15.305615 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.305576    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istiod-ca-cert\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.305798 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.305621    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-envoy\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.305798 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.305647    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95rjq\" (UniqueName: \"kubernetes.io/projected/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-kube-api-access-95rjq\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.305798 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.305692    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-workload-certs\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.305798 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.305784    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-credential-socket\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.306011 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.305852    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-podinfo\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.306011 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.305947    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-token\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.306103 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.306010    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-data\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.306145 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.306107    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-workload-socket\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.407547 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.407504    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-credential-socket\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.407764 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.407607    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-podinfo\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.407764 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.407663    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-token\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.407764 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.407715    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-data\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.407933 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.407766    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-workload-socket\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.407933 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.407798    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istiod-ca-cert\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.407933 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.407830    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-envoy\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.407933 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.407854    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95rjq\" (UniqueName: \"kubernetes.io/projected/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-kube-api-access-95rjq\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.407933 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.407884    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-workload-certs\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.408184 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.407962    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-credential-socket\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.408184 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.408085    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-data\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.408184 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.408163    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-workload-socket\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.408327 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.408250    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-workload-certs\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.408579 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.408541    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istiod-ca-cert\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.410048 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.410026    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-envoy\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.410169 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.410148    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-podinfo\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.422095 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.422041    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-token\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.425065 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.425031    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95rjq\" (UniqueName: \"kubernetes.io/projected/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-kube-api-access-95rjq\") pod \"e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") " pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.482773 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.482736    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:15.697078 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.696986    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:15.765707 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.765606    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"]
Sep 15 11:40:15.767828 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:40:15.767803    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8910796a_392d_4dbe_8ae9_cd70d7cdc4c0.slice/crio-102fc2b0d6c5fa75dc6e30341b488bd5a08b839db4e4c91ae28ea99d47c9396b WatchSource:0}: Error finding container 102fc2b0d6c5fa75dc6e30341b488bd5a08b839db4e4c91ae28ea99d47c9396b: Status 404 returned error can't find the container with id 102fc2b0d6c5fa75dc6e30341b488bd5a08b839db4e4c91ae28ea99d47c9396b
Sep 15 11:40:15.770348 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.770314    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:40:15.770430 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.770384    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:40:15.770430 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.770413    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:40:15.999817 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:15.999702    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:16.563835 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:16.563797    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n" event={"ID":"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0","Type":"ContainerStarted","Data":"99eef4216eea4554bcf4a068163e093b0b8b083048aa3bee1d0f086b498cee35"}
Sep 15 11:40:16.564198 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:16.563855    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n" event={"ID":"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0","Type":"ContainerStarted","Data":"102fc2b0d6c5fa75dc6e30341b488bd5a08b839db4e4c91ae28ea99d47c9396b"}
Sep 15 11:40:16.589764 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:16.589707    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n" podStartSLOduration=1.589689995 podStartE2EDuration="1.589689995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:40:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:40:16.588048849 +0000 UTC m=+918.080826009" watchObservedRunningTime="2026-09-15 11:40:16.589689995 +0000 UTC m=+918.082467200"
Sep 15 11:40:16.905507 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:16.905475    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:17.483689 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:17.483646    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:17.490075 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:17.490045    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:17.570101 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:17.570068    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:17.571072 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:17.571050    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:17.786748 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:17.786633    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:18.029194 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:18.029155    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:18.130116 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:18.130080    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:18.281041 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:18.281006    2560 patch_prober.go:28] interesting pod/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 11:40:18.281206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:18.281068    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g" podUID="ba61d864-c491-4420-8b4b-7dd33b6947c5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 11:40:18.386109 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:18.386022    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:18.583004 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:18.582971    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:19.518556 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.518524    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:19.588551 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.587417    2560 generic.go:356] "Generic (PLEG): container finished" podID="ba61d864-c491-4420-8b4b-7dd33b6947c5" containerID="d907324b13d3fc4afa027dc99eb429c5db1310ab58b018fb25e24a1249ec0411" exitCode=0
Sep 15 11:40:19.588551 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.588349    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g" event={"ID":"ba61d864-c491-4420-8b4b-7dd33b6947c5","Type":"ContainerDied","Data":"d907324b13d3fc4afa027dc99eb429c5db1310ab58b018fb25e24a1249ec0411"}
Sep 15 11:40:19.588551 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.588408    2560 scope.go:117] "RemoveContainer" containerID="d61025ff75996ea396aad62a6e9e63c246660fb889f73ed721f24e511c7b3ec9"
Sep 15 11:40:19.705910 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.705847    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:40:19.857140 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.857109    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-envoy\") pod \"ba61d864-c491-4420-8b4b-7dd33b6947c5\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") "
Sep 15 11:40:19.857140 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.857147    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-workload-socket\") pod \"ba61d864-c491-4420-8b4b-7dd33b6947c5\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") "
Sep 15 11:40:19.857354 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.857176    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-workload-certs\") pod \"ba61d864-c491-4420-8b4b-7dd33b6947c5\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") "
Sep 15 11:40:19.857354 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.857193    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba61d864-c491-4420-8b4b-7dd33b6947c5-istiod-ca-cert\") pod \"ba61d864-c491-4420-8b4b-7dd33b6947c5\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") "
Sep 15 11:40:19.857354 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.857211    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-podinfo\") pod \"ba61d864-c491-4420-8b4b-7dd33b6947c5\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") "
Sep 15 11:40:19.857354 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.857250    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-credential-socket\") pod \"ba61d864-c491-4420-8b4b-7dd33b6947c5\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") "
Sep 15 11:40:19.857354 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.857276    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-token\") pod \"ba61d864-c491-4420-8b4b-7dd33b6947c5\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") "
Sep 15 11:40:19.857640 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.857492    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ba61d864-c491-4420-8b4b-7dd33b6947c5" (UID: "ba61d864-c491-4420-8b4b-7dd33b6947c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:40:19.857640 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.857502    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ba61d864-c491-4420-8b4b-7dd33b6947c5" (UID: "ba61d864-c491-4420-8b4b-7dd33b6947c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:40:19.857640 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.857572    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ba61d864-c491-4420-8b4b-7dd33b6947c5" (UID: "ba61d864-c491-4420-8b4b-7dd33b6947c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:40:19.857798 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.857642    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba61d864-c491-4420-8b4b-7dd33b6947c5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ba61d864-c491-4420-8b4b-7dd33b6947c5" (UID: "ba61d864-c491-4420-8b4b-7dd33b6947c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:40:19.857798 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.857656    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9whhh\" (UniqueName: \"kubernetes.io/projected/ba61d864-c491-4420-8b4b-7dd33b6947c5-kube-api-access-9whhh\") pod \"ba61d864-c491-4420-8b4b-7dd33b6947c5\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") "
Sep 15 11:40:19.857798 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.857745    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-data\") pod \"ba61d864-c491-4420-8b4b-7dd33b6947c5\" (UID: \"ba61d864-c491-4420-8b4b-7dd33b6947c5\") "
Sep 15 11:40:19.858010 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.857987    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-data" (OuterVolumeSpecName: "istio-data") pod "ba61d864-c491-4420-8b4b-7dd33b6947c5" (UID: "ba61d864-c491-4420-8b4b-7dd33b6947c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:40:19.858166 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.858141    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:40:19.858230 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.858173    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:40:19.858230 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.858188    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:40:19.858230 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.858204    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:40:19.858230 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.858218    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba61d864-c491-4420-8b4b-7dd33b6947c5-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:40:19.859699 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.859677    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-token" (OuterVolumeSpecName: "istio-token") pod "ba61d864-c491-4420-8b4b-7dd33b6947c5" (UID: "ba61d864-c491-4420-8b4b-7dd33b6947c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:40:19.859818 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.859740    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba61d864-c491-4420-8b4b-7dd33b6947c5-kube-api-access-9whhh" (OuterVolumeSpecName: "kube-api-access-9whhh") pod "ba61d864-c491-4420-8b4b-7dd33b6947c5" (UID: "ba61d864-c491-4420-8b4b-7dd33b6947c5"). InnerVolumeSpecName "kube-api-access-9whhh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:40:19.859875 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.859858    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ba61d864-c491-4420-8b4b-7dd33b6947c5" (UID: "ba61d864-c491-4420-8b4b-7dd33b6947c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:40:19.859918 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.859866    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ba61d864-c491-4420-8b4b-7dd33b6947c5" (UID: "ba61d864-c491-4420-8b4b-7dd33b6947c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:40:19.959598 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.959503    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:40:19.959598 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.959528    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:40:19.959598 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.959537    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba61d864-c491-4420-8b4b-7dd33b6947c5-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:40:19.959598 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:19.959545    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9whhh\" (UniqueName: \"kubernetes.io/projected/ba61d864-c491-4420-8b4b-7dd33b6947c5-kube-api-access-9whhh\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:40:20.184183 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:20.184143    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:20.596386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:20.596234    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g" event={"ID":"ba61d864-c491-4420-8b4b-7dd33b6947c5","Type":"ContainerDied","Data":"264042f0bd6346df38ef1604e3117612de15c32e2e15a457bced84d0317b6225"}
Sep 15 11:40:20.596386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:20.596283    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"
Sep 15 11:40:20.596386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:20.596305    2560 scope.go:117] "RemoveContainer" containerID="d907324b13d3fc4afa027dc99eb429c5db1310ab58b018fb25e24a1249ec0411"
Sep 15 11:40:20.620126 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:20.620101    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"]
Sep 15 11:40:20.627168 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:20.627144    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9b1cbda7-openshift-default-5d96788786-d9j8g"]
Sep 15 11:40:21.036177 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:21.036089    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba61d864-c491-4420-8b4b-7dd33b6947c5" path="/var/lib/kubelet/pods/ba61d864-c491-4420-8b4b-7dd33b6947c5/volumes"
Sep 15 11:40:21.223700 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:21.223655    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:22.074645 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:22.074606    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:22.175926 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:22.175893    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:22.274411 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:22.274377    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:22.510655 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:22.510624    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:22.618804 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:22.618771    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:26.436901 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:26.436867    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:43.737857 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:43.737707    2560 generic.go:356] "Generic (PLEG): container finished" podID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" containerID="99eef4216eea4554bcf4a068163e093b0b8b083048aa3bee1d0f086b498cee35" exitCode=0
Sep 15 11:40:43.737857 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:43.737783    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n" event={"ID":"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0","Type":"ContainerDied","Data":"99eef4216eea4554bcf4a068163e093b0b8b083048aa3bee1d0f086b498cee35"}
Sep 15 11:40:43.738411 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:43.738285    2560 scope.go:117] "RemoveContainer" containerID="99eef4216eea4554bcf4a068163e093b0b8b083048aa3bee1d0f086b498cee35"
Sep 15 11:40:43.739494 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:43.739461    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:40:43.739636 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:43.739542    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:40:43.739636 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:43.739605    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:40:44.098545 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:44.098508    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:44.745267 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:44.745232    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n" event={"ID":"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0","Type":"ContainerStarted","Data":"c1985b3b36191625943588927e048fef1f9efd5e6123cdc6a3ce230eae94fe0f"}
Sep 15 11:40:44.823118 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:44.823079    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:45.483396 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:45.483363    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:45.484853 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:45.484825    2560 patch_prober.go:28] interesting pod/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 15 11:40:45.484982 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:45.484881    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n" podUID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 15 11:40:45.782648 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:45.782540    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:46.483155 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:46.483120    2560 patch_prober.go:28] interesting pod/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 15 11:40:46.483332 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:46.483183    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n" podUID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 15 11:40:46.598207 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:46.598161    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:47.435143 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:47.435099    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:47.483413 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:47.483376    2560 patch_prober.go:28] interesting pod/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 15 11:40:47.483620 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:47.483472    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n" podUID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 15 11:40:47.570397 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:47.570362    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:48.483789 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:48.483750    2560 patch_prober.go:28] interesting pod/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 15 11:40:48.484173 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:48.483808    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n" podUID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 15 11:40:48.626660 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:48.626621    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:49.437956 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:49.437914    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:49.484227 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:49.484072    2560 patch_prober.go:28] interesting pod/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 15 11:40:49.484227 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:49.484152    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n" podUID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 15 11:40:50.213520 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:50.213476    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:50.483321 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:50.483231    2560 patch_prober.go:28] interesting pod/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 15 11:40:50.483321 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:50.483289    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n" podUID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 15 11:40:51.300275 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:51.300235    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:51.488016 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:51.487985    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:51.489171 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:51.489148    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:40:52.116494 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:52.116447    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:52.942162 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:52.942124    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:54.627809 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:54.627777    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:55.389759 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:55.389718    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:40:59.390976 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:40:59.390939    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:00.076380 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:00.076343    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:00.170500 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:00.170447    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:00.233807 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:00.233773    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:00.297597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:00.297546    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:00.417583 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:00.417534    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"]
Sep 15 11:41:00.418067 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:00.417809    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n" podUID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" containerName="istio-proxy" containerID="cri-o://c1985b3b36191625943588927e048fef1f9efd5e6123cdc6a3ce230eae94fe0f" gracePeriod=30
Sep 15 11:41:00.441483 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:00.441439    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:00.757143 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:00.757067    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:01.106178 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:01.106145    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:01.205290 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:01.205241    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:01.305227 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:01.305191    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:01.849245 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:01.849217    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:02.495253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:02.495199    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:02.745024 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:02.744988    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:03.172962 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:03.172924    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:03.986415 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:03.986371    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:05.781015 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.780990    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:41:05.868905 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.868868    2560 generic.go:356] "Generic (PLEG): container finished" podID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" containerID="c1985b3b36191625943588927e048fef1f9efd5e6123cdc6a3ce230eae94fe0f" exitCode=0
Sep 15 11:41:05.869053 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.868917    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"
Sep 15 11:41:05.869053 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.868947    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n" event={"ID":"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0","Type":"ContainerDied","Data":"c1985b3b36191625943588927e048fef1f9efd5e6123cdc6a3ce230eae94fe0f"}
Sep 15 11:41:05.869053 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.869001    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n" event={"ID":"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0","Type":"ContainerDied","Data":"102fc2b0d6c5fa75dc6e30341b488bd5a08b839db4e4c91ae28ea99d47c9396b"}
Sep 15 11:41:05.869053 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.869024    2560 scope.go:117] "RemoveContainer" containerID="c1985b3b36191625943588927e048fef1f9efd5e6123cdc6a3ce230eae94fe0f"
Sep 15 11:41:05.878127 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.878111    2560 scope.go:117] "RemoveContainer" containerID="99eef4216eea4554bcf4a068163e093b0b8b083048aa3bee1d0f086b498cee35"
Sep 15 11:41:05.886103 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.886083    2560 scope.go:117] "RemoveContainer" containerID="c1985b3b36191625943588927e048fef1f9efd5e6123cdc6a3ce230eae94fe0f"
Sep 15 11:41:05.886348 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:41:05.886333    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1985b3b36191625943588927e048fef1f9efd5e6123cdc6a3ce230eae94fe0f\": container with ID starting with c1985b3b36191625943588927e048fef1f9efd5e6123cdc6a3ce230eae94fe0f not found: ID does not exist" containerID="c1985b3b36191625943588927e048fef1f9efd5e6123cdc6a3ce230eae94fe0f"
Sep 15 11:41:05.886402 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.886358    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1985b3b36191625943588927e048fef1f9efd5e6123cdc6a3ce230eae94fe0f"} err="failed to get container status \"c1985b3b36191625943588927e048fef1f9efd5e6123cdc6a3ce230eae94fe0f\": rpc error: code = NotFound desc = could not find container \"c1985b3b36191625943588927e048fef1f9efd5e6123cdc6a3ce230eae94fe0f\": container with ID starting with c1985b3b36191625943588927e048fef1f9efd5e6123cdc6a3ce230eae94fe0f not found: ID does not exist"
Sep 15 11:41:05.886402 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.886377    2560 scope.go:117] "RemoveContainer" containerID="99eef4216eea4554bcf4a068163e093b0b8b083048aa3bee1d0f086b498cee35"
Sep 15 11:41:05.886618 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:41:05.886597    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99eef4216eea4554bcf4a068163e093b0b8b083048aa3bee1d0f086b498cee35\": container with ID starting with 99eef4216eea4554bcf4a068163e093b0b8b083048aa3bee1d0f086b498cee35 not found: ID does not exist" containerID="99eef4216eea4554bcf4a068163e093b0b8b083048aa3bee1d0f086b498cee35"
Sep 15 11:41:05.886700 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.886625    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99eef4216eea4554bcf4a068163e093b0b8b083048aa3bee1d0f086b498cee35"} err="failed to get container status \"99eef4216eea4554bcf4a068163e093b0b8b083048aa3bee1d0f086b498cee35\": rpc error: code = NotFound desc = could not find container \"99eef4216eea4554bcf4a068163e093b0b8b083048aa3bee1d0f086b498cee35\": container with ID starting with 99eef4216eea4554bcf4a068163e093b0b8b083048aa3bee1d0f086b498cee35 not found: ID does not exist"
Sep 15 11:41:05.928109 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928086    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istiod-ca-cert\") pod \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") "
Sep 15 11:41:05.928213 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928138    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-token\") pod \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") "
Sep 15 11:41:05.928213 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928159    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-data\") pod \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") "
Sep 15 11:41:05.928292 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928226    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-credential-socket\") pod \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") "
Sep 15 11:41:05.928292 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928276    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-workload-socket\") pod \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") "
Sep 15 11:41:05.928395 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928303    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-workload-certs\") pod \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") "
Sep 15 11:41:05.928395 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928354    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-podinfo\") pod \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") "
Sep 15 11:41:05.928505 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928410    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" (UID: "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:41:05.928505 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928433    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-envoy\") pod \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") "
Sep 15 11:41:05.928505 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928444    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" (UID: "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:41:05.928505 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928490    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95rjq\" (UniqueName: \"kubernetes.io/projected/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-kube-api-access-95rjq\") pod \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\" (UID: \"8910796a-392d-4dbe-8ae9-cd70d7cdc4c0\") "
Sep 15 11:41:05.928717 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928528    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" (UID: "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:41:05.928717 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928631    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" (UID: "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:41:05.928821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928782    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" (UID: "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:41:05.928919 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928901    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:41:05.928989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928923    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:41:05.928989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928936    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:41:05.928989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928949    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:41:05.928989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.928963    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:41:05.930317 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.930295    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" (UID: "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:41:05.930411 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.930392    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" (UID: "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:41:05.930496 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.930477    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" (UID: "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:41:05.930640 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:05.930543    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-kube-api-access-95rjq" (OuterVolumeSpecName: "kube-api-access-95rjq") pod "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" (UID: "8910796a-392d-4dbe-8ae9-cd70d7cdc4c0"). InnerVolumeSpecName "kube-api-access-95rjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:41:06.029409 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:06.029384    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:41:06.029409 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:06.029406    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:41:06.029586 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:06.029416    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:41:06.029586 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:06.029424    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95rjq\" (UniqueName: \"kubernetes.io/projected/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0-kube-api-access-95rjq\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:41:06.192298 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:06.192267    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"]
Sep 15 11:41:06.195659 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:06.195635    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-6f21996a-openshift-default-5d6f9b5ff6-trf6n"]
Sep 15 11:41:07.034902 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:07.034869    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" path="/var/lib/kubelet/pods/8910796a-392d-4dbe-8ae9-cd70d7cdc4c0/volumes"
Sep 15 11:41:08.693746 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:08.693690    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:08.796394 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:08.796352    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:08.898420 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:08.898371    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:08.972616 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:08.972505    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:09.443092 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:09.443060    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:10.959276 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:10.959233    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:11.058909 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:11.058871    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:11.229601 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:11.229496    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:11.648451 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:11.648415    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:12.081197 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.081123    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"]
Sep 15 11:41:12.081582 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.081532    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba61d864-c491-4420-8b4b-7dd33b6947c5" containerName="istio-proxy"
Sep 15 11:41:12.081582 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.081543    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba61d864-c491-4420-8b4b-7dd33b6947c5" containerName="istio-proxy"
Sep 15 11:41:12.081582 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.081551    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" containerName="istio-proxy"
Sep 15 11:41:12.081582 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.081557    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" containerName="istio-proxy"
Sep 15 11:41:12.081729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.081594    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" containerName="istio-proxy"
Sep 15 11:41:12.081729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.081600    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" containerName="istio-proxy"
Sep 15 11:41:12.081729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.081672    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" containerName="istio-proxy"
Sep 15 11:41:12.081729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.081680    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8910796a-392d-4dbe-8ae9-cd70d7cdc4c0" containerName="istio-proxy"
Sep 15 11:41:12.081729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.081687    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba61d864-c491-4420-8b4b-7dd33b6947c5" containerName="istio-proxy"
Sep 15 11:41:12.081729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.081695    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba61d864-c491-4420-8b4b-7dd33b6947c5" containerName="istio-proxy"
Sep 15 11:41:12.086215 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.086191    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.088756 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.088734    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-a73e48-openshift-default-dockercfg-fvh8r\""
Sep 15 11:41:12.110034 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.110008    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"]
Sep 15 11:41:12.118264 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.118239    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:12.186489 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.186452    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-workload-socket\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.186708 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.186512    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-workload-certs\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.186708 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.186550    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-data\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.186708 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.186597    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmgvs\" (UniqueName: \"kubernetes.io/projected/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-kube-api-access-pmgvs\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.186708 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.186629    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-podinfo\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.186708 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.186657    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-token\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.186708 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.186693    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-envoy\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.187061 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.186743    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istiod-ca-cert\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.187061 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.186780    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-credential-socket\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.267496 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.267463    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:12.287751 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.287723    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-credential-socket\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.287917 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.287839    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-workload-socket\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.287917 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.287888    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-workload-certs\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.288028 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.287928    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-data\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.288028 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.287963    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmgvs\" (UniqueName: \"kubernetes.io/projected/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-kube-api-access-pmgvs\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.288028 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.287993    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-podinfo\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.288028 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.288022    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-token\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.288233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.288059    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-envoy\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.288233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.288118    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istiod-ca-cert\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.288233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.288171    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-credential-socket\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.288884 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.288534    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-workload-certs\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.288884 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.288826    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istiod-ca-cert\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.289085 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.289051    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-data\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.289146 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.289087    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-workload-socket\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.291290 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.291263    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-envoy\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.291394 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.291274    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-podinfo\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.296509 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.296479    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-token\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.296721 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.296701    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmgvs\" (UniqueName: \"kubernetes.io/projected/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-kube-api-access-pmgvs\") pod \"e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") " pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.398462 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.398435    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:12.680656 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.680628    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"]
Sep 15 11:41:12.682802 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:41:12.682775    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bd60ba0_bdce_448c_89bf_6ad6dd292a10.slice/crio-8ee6874474efa4fd2d79a7f4299809759b247cbfd0b4482e9a276fbbb66da855 WatchSource:0}: Error finding container 8ee6874474efa4fd2d79a7f4299809759b247cbfd0b4482e9a276fbbb66da855: Status 404 returned error can't find the container with id 8ee6874474efa4fd2d79a7f4299809759b247cbfd0b4482e9a276fbbb66da855
Sep 15 11:41:12.685396 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.685345    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:41:12.685493 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.685444    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:41:12.685628 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.685513    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:41:12.912274 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.912229    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs" event={"ID":"8bd60ba0-bdce-448c-89bf-6ad6dd292a10","Type":"ContainerStarted","Data":"651afd7300778c512899d28bd317cdc89b089bd8777a1bc6ca8c41232e32e9f3"}
Sep 15 11:41:12.912449 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.912298    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs" event={"ID":"8bd60ba0-bdce-448c-89bf-6ad6dd292a10","Type":"ContainerStarted","Data":"8ee6874474efa4fd2d79a7f4299809759b247cbfd0b4482e9a276fbbb66da855"}
Sep 15 11:41:12.937655 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:12.937544    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs" podStartSLOduration=0.937529416 podStartE2EDuration="937.529416ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:41:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:41:12.93508789 +0000 UTC m=+974.427865037" watchObservedRunningTime="2026-09-15 11:41:12.937529416 +0000 UTC m=+974.430306564"
Sep 15 11:41:13.399479 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:13.399442    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:13.405161 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:13.405136    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:13.918601 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:13.918540    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:13.919514 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:13.919492    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:14.185116 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:14.185017    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:14.865688 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:14.865640    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:15.013373 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:15.013298    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:15.251297 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:15.251185    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:16.058293 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:16.058260    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:16.821120 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:16.821083    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:17.705365 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:17.705319    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:18.502598 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:18.502551    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:18.610967 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:18.610937    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:18.933477 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:18.933446    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:19.172084 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.172052    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:19.266571 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.266464    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:19.362715 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.362679    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"]
Sep 15 11:41:19.363140 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.363126    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba61d864-c491-4420-8b4b-7dd33b6947c5" containerName="istio-proxy"
Sep 15 11:41:19.363202 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.363143    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba61d864-c491-4420-8b4b-7dd33b6947c5" containerName="istio-proxy"
Sep 15 11:41:19.366485 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.366468    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"
Sep 15 11:41:19.369135 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.369106    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-a73e48-dockercfg-cp5mq\""
Sep 15 11:41:19.369650 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.369631    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-a73e48\""
Sep 15 11:41:19.385038 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.385013    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"]
Sep 15 11:41:19.454321 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.454284    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjv2b\" (UniqueName: \"kubernetes.io/projected/5418c25a-f580-47e0-b34b-afabee439353-kube-api-access-xjv2b\") pod \"payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq\" (UID: \"5418c25a-f580-47e0-b34b-afabee439353\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"
Sep 15 11:41:19.454642 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.454621    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5418c25a-f580-47e0-b34b-afabee439353-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq\" (UID: \"5418c25a-f580-47e0-b34b-afabee439353\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"
Sep 15 11:41:19.555632 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.555517    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjv2b\" (UniqueName: \"kubernetes.io/projected/5418c25a-f580-47e0-b34b-afabee439353-kube-api-access-xjv2b\") pod \"payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq\" (UID: \"5418c25a-f580-47e0-b34b-afabee439353\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"
Sep 15 11:41:19.555632 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.555583    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5418c25a-f580-47e0-b34b-afabee439353-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq\" (UID: \"5418c25a-f580-47e0-b34b-afabee439353\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"
Sep 15 11:41:19.556355 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.556335    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5418c25a-f580-47e0-b34b-afabee439353-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq\" (UID: \"5418c25a-f580-47e0-b34b-afabee439353\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"
Sep 15 11:41:19.565475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.565448    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjv2b\" (UniqueName: \"kubernetes.io/projected/5418c25a-f580-47e0-b34b-afabee439353-kube-api-access-xjv2b\") pod \"payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq\" (UID: \"5418c25a-f580-47e0-b34b-afabee439353\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"
Sep 15 11:41:19.678388 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.678349    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"
Sep 15 11:41:19.972201 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:19.972096    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"]
Sep 15 11:41:19.975095 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:41:19.975065    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5418c25a_f580_47e0_b34b_afabee439353.slice/crio-bfe183050d9a9771d434ea15cabe32aa69e6da44ee675b940b68c910d761a4c8 WatchSource:0}: Error finding container bfe183050d9a9771d434ea15cabe32aa69e6da44ee675b940b68c910d761a4c8: Status 404 returned error can't find the container with id bfe183050d9a9771d434ea15cabe32aa69e6da44ee675b940b68c910d761a4c8
Sep 15 11:41:20.957504 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:20.957471    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq" event={"ID":"5418c25a-f580-47e0-b34b-afabee439353","Type":"ContainerStarted","Data":"b51df9ec34a2a214d228030f5d7373d86d5570c5f74a816187f2f5fb9ca1e5d4"}
Sep 15 11:41:20.957709 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:20.957528    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"
Sep 15 11:41:20.957709 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:20.957550    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq" event={"ID":"5418c25a-f580-47e0-b34b-afabee439353","Type":"ContainerStarted","Data":"bfe183050d9a9771d434ea15cabe32aa69e6da44ee675b940b68c910d761a4c8"}
Sep 15 11:41:20.981123 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:20.981054    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq" podStartSLOduration=1.981033298 podStartE2EDuration="1.981033298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:41:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:41:20.978271574 +0000 UTC m=+982.471048726" watchObservedRunningTime="2026-09-15 11:41:20.981033298 +0000 UTC m=+982.473810447"
Sep 15 11:41:31.965734 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:31.965702    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"
Sep 15 11:41:35.508684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:35.508637    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:39.062219 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:39.062125    2560 generic.go:356] "Generic (PLEG): container finished" podID="8bd60ba0-bdce-448c-89bf-6ad6dd292a10" containerID="651afd7300778c512899d28bd317cdc89b089bd8777a1bc6ca8c41232e32e9f3" exitCode=0
Sep 15 11:41:39.062219 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:39.062180    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs" event={"ID":"8bd60ba0-bdce-448c-89bf-6ad6dd292a10","Type":"ContainerDied","Data":"651afd7300778c512899d28bd317cdc89b089bd8777a1bc6ca8c41232e32e9f3"}
Sep 15 11:41:39.062715 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:39.062585    2560 scope.go:117] "RemoveContainer" containerID="651afd7300778c512899d28bd317cdc89b089bd8777a1bc6ca8c41232e32e9f3"
Sep 15 11:41:39.063646 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:39.063618    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:41:39.063704 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:39.063684    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:41:39.063742 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:39.063716    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:41:40.070554 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:40.070519    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs" event={"ID":"8bd60ba0-bdce-448c-89bf-6ad6dd292a10","Type":"ContainerStarted","Data":"2c0e02abbfbaae2f60fca18e57da8c0daf3758510243e8774f95c64f5344929b"}
Sep 15 11:41:40.400342 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:40.400299    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:40.400576 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:40.400502    2560 patch_prober.go:28] interesting pod/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 15 11:41:40.400652 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:40.400552    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs" podUID="8bd60ba0-bdce-448c-89bf-6ad6dd292a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 15 11:41:40.992716 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:40.992682    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:41.340195 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:41.340152    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:41.399710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:41.399674    2560 patch_prober.go:28] interesting pod/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 15 11:41:41.400005 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:41.399972    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs" podUID="8bd60ba0-bdce-448c-89bf-6ad6dd292a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 15 11:41:41.452469 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:41.452435    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:41.538976 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:41.538909    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:42.033521 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:42.033483    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:42.399654 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:42.399611    2560 patch_prober.go:28] interesting pod/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 15 11:41:42.400328 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:42.400286    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs" podUID="8bd60ba0-bdce-448c-89bf-6ad6dd292a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 15 11:41:43.077361 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:43.077319    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:43.399195 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:43.399159    2560 patch_prober.go:28] interesting pod/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 15 11:41:43.399476 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:43.399448    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs" podUID="8bd60ba0-bdce-448c-89bf-6ad6dd292a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 15 11:41:43.895327 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:43.895292    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:43.919127 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:43.919105    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:44.400606 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:44.399623    2560 patch_prober.go:28] interesting pod/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 15 11:41:44.400606 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:44.399684    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs" podUID="8bd60ba0-bdce-448c-89bf-6ad6dd292a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 15 11:41:44.566049 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:44.566010    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:44.918043 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:44.918005    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:45.411692 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:45.411653    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:45.428010 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:45.427977    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:45.429891 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:45.429855    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:41:55.931893 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:55.931845    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:56.033504 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:56.033466    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:57.412577 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:57.412537    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:41:58.691709 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:41:58.691674    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:03.225452 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:03.225404    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:03.331688 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:03.331652    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:03.434573 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:03.434537    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:13.939090 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:13.939023    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:14.726573 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:14.726522    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:20.442632 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:20.442594    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:20.509621 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:20.509581    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:20.631622 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:20.631586    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:20.727635 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:20.727527    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:21.396780 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:21.396745    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:22.178712 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:22.178674    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:22.282883 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:22.282828    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:23.103080 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:23.103047    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:27.266158 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:27.266116    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:29.031113 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:29.031077    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:29.127795 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:29.127760    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:29.719960 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:29.719928    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:33.143347 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:33.143313    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:35.520992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:35.520956    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:35.622183 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:35.622150    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:35.739159 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:35.739128    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:36.021281 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.021246    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"]
Sep 15 11:42:36.025343 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.025322    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.027649 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.027628    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5f7ca109-openshift-default-dockercfg-nwg57\""
Sep 15 11:42:36.038062 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.038033    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"]
Sep 15 11:42:36.053005 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.052975    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:36.101831 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.101799    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-data\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.101993 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.101852    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-token\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.101993 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.101884    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-envoy\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.101993 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.101970    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4t9w\" (UniqueName: \"kubernetes.io/projected/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-kube-api-access-c4t9w\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.102174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.102051    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-workload-certs\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.102174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.102087    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-podinfo\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.102174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.102128    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istiod-ca-cert\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.102174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.102161    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-workload-socket\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.102360 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.102189    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-credential-socket\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.202833 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.202751    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-token\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.202833 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.202795    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-envoy\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.203063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.202900    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4t9w\" (UniqueName: \"kubernetes.io/projected/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-kube-api-access-c4t9w\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.203063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.202944    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-workload-certs\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.203063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.202965    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-podinfo\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.203063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.202990    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istiod-ca-cert\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.203276 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.203144    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-workload-socket\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.203276 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.203191    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-credential-socket\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.203276 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.203264    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-data\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.203464 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.203431    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-workload-certs\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.203547 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.203521    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-workload-socket\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.203654 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.203634    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-data\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.203801 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.203779    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istiod-ca-cert\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.203926 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.203904    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-credential-socket\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.205228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.205198    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-envoy\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.205539 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.205519    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-podinfo\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.210794 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.210764    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-token\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.211003 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.210980    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4t9w\" (UniqueName: \"kubernetes.io/projected/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-kube-api-access-c4t9w\") pod \"e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") " pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.336050 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.336019    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:36.368126 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.368083    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:36.654397 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:42:36.654365    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e36e9af_a80b_4e01_b1f6_844ef3040a7f.slice/crio-c84a862ef66f35bdde091d269610c11b200ada0fd12f61a03946ae79526e35e7 WatchSource:0}: Error finding container c84a862ef66f35bdde091d269610c11b200ada0fd12f61a03946ae79526e35e7: Status 404 returned error can't find the container with id c84a862ef66f35bdde091d269610c11b200ada0fd12f61a03946ae79526e35e7
Sep 15 11:42:36.661602 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.657315    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:42:36.661602 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.657526    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:42:36.661602 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.657634    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:42:36.676632 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:36.676609    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"]
Sep 15 11:42:37.384071 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:37.384031    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb" event={"ID":"8e36e9af-a80b-4e01-b1f6-844ef3040a7f","Type":"ContainerStarted","Data":"7a3f3a007c0bae10679b176fa3c70df8d994d50e275995a916b676fe1596e1e0"}
Sep 15 11:42:37.384273 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:37.384092    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb" event={"ID":"8e36e9af-a80b-4e01-b1f6-844ef3040a7f","Type":"ContainerStarted","Data":"c84a862ef66f35bdde091d269610c11b200ada0fd12f61a03946ae79526e35e7"}
Sep 15 11:42:37.410035 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:37.409986    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb" podStartSLOduration=2.409971922 podStartE2EDuration="2.409971922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:42:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:42:37.407989251 +0000 UTC m=+1058.900766399" watchObservedRunningTime="2026-09-15 11:42:37.409971922 +0000 UTC m=+1058.902749070"
Sep 15 11:42:38.337177 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:38.337136    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:38.342599 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:38.342574    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:38.390438 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:38.390408    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:38.391494 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:38.391472    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:42:38.433381 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:38.433345    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:38.521914 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:38.521880    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:41.440168 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:41.440130    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:42.241708 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:42.241673    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:42.539074 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:42.538994    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:42.994614 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:42.994555    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:43.939362 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:43.939321    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:44.893636 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:44.893596    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:45.698369 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:45.698313    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:46.580080 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:46.580044    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:47.408671 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:47.408640    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:47.484260 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:47.484215    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:47.748197 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:47.748098    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:47.872660 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:47.872625    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:47.972034 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:47.972001    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:54.637239 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:54.637158    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:56.980706 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:56.980662    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:57.069982 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:57.069947    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz"]
Sep 15 11:42:57.073784 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:57.073765    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz"
Sep 15 11:42:57.076446 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:57.076425    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbxjk\""
Sep 15 11:42:57.089520 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:57.089495    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz"]
Sep 15 11:42:57.118015 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:57.117977    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66f17071-39b0-406b-bcf5-462b1b512241-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz\" (UID: \"66f17071-39b0-406b-bcf5-462b1b512241\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz"
Sep 15 11:42:57.212808 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:57.212775    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:57.218491 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:57.218468    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66f17071-39b0-406b-bcf5-462b1b512241-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz\" (UID: \"66f17071-39b0-406b-bcf5-462b1b512241\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz"
Sep 15 11:42:57.219124 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:57.219102    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66f17071-39b0-406b-bcf5-462b1b512241-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz\" (UID: \"66f17071-39b0-406b-bcf5-462b1b512241\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz"
Sep 15 11:42:57.384729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:57.384698    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz"
Sep 15 11:42:57.659165 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:57.659087    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz"]
Sep 15 11:42:57.661452 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:42:57.661425    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66f17071_39b0_406b_bcf5_462b1b512241.slice/crio-51f607f7a7a9784f810562e6689033e2fd9f96167981a3a19d8b6a809deb6e8d WatchSource:0}: Error finding container 51f607f7a7a9784f810562e6689033e2fd9f96167981a3a19d8b6a809deb6e8d: Status 404 returned error can't find the container with id 51f607f7a7a9784f810562e6689033e2fd9f96167981a3a19d8b6a809deb6e8d
Sep 15 11:42:58.003381 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:58.003350    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:58.509404 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:58.509377    2560 generic.go:356] "Generic (PLEG): container finished" podID="66f17071-39b0-406b-bcf5-462b1b512241" containerID="9b91e34a1473027f58cf7f670a52eab1ec44b8f87c3e42ba544b2b983989e5ec" exitCode=0
Sep 15 11:42:58.509536 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:58.509445    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz" event={"ID":"66f17071-39b0-406b-bcf5-462b1b512241","Type":"ContainerDied","Data":"9b91e34a1473027f58cf7f670a52eab1ec44b8f87c3e42ba544b2b983989e5ec"}
Sep 15 11:42:58.509536 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:58.509495    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz" event={"ID":"66f17071-39b0-406b-bcf5-462b1b512241","Type":"ContainerStarted","Data":"51f607f7a7a9784f810562e6689033e2fd9f96167981a3a19d8b6a809deb6e8d"}
Sep 15 11:42:59.057464 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:59.057420    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:42:59.701594 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:59.700913    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz"
Sep 15 11:42:59.746376 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:59.746030    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66f17071-39b0-406b-bcf5-462b1b512241-maas-api-service-ca\") pod \"66f17071-39b0-406b-bcf5-462b1b512241\" (UID: \"66f17071-39b0-406b-bcf5-462b1b512241\") "
Sep 15 11:42:59.746750 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:59.746719    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f17071-39b0-406b-bcf5-462b1b512241-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66f17071-39b0-406b-bcf5-462b1b512241" (UID: "66f17071-39b0-406b-bcf5-462b1b512241"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:42:59.848595 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:42:59.848243    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66f17071-39b0-406b-bcf5-462b1b512241-maas-api-service-ca\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:00.066769 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:00.066734    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:00.528265 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:00.528239    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz"
Sep 15 11:43:00.528265 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:00.528254    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz" event={"ID":"66f17071-39b0-406b-bcf5-462b1b512241","Type":"ContainerDied","Data":"51f607f7a7a9784f810562e6689033e2fd9f96167981a3a19d8b6a809deb6e8d"}
Sep 15 11:43:00.528475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:00.528298    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51f607f7a7a9784f810562e6689033e2fd9f96167981a3a19d8b6a809deb6e8d"
Sep 15 11:43:01.025797 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:01.025750    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:01.827084 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:01.827052    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:01.927742 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:01.927709    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:02.621152 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:02.621116    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:02.722533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:02.722495    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:02.854466 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:02.854420    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:03.045689 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:03.045609    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:03.700907 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:03.700819    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:04.866491 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:04.866457    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:05.597875 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:05.597830    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:06.378435 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:06.378393    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:07.203850 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:07.203807    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:07.786413 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:07.786379    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:07.867930 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:07.867835    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"]
Sep 15 11:43:07.870151 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:07.868145    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq" podUID="5418c25a-f580-47e0-b34b-afabee439353" containerName="payload-pre-processing" containerID="cri-o://b51df9ec34a2a214d228030f5d7373d86d5570c5f74a816187f2f5fb9ca1e5d4" gracePeriod=30
Sep 15 11:43:07.895022 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:07.894988    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:07.992700 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:07.992646    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:08.081235 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.081159    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz"]
Sep 15 11:43:08.082791 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.082653    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:08.083706 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.083686    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a73e48-4vlzz"]
Sep 15 11:43:08.137723 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.137701    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"
Sep 15 11:43:08.178604 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.177912    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:08.239268 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.239231    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5418c25a-f580-47e0-b34b-afabee439353-plugins-config-volume\") pod \"5418c25a-f580-47e0-b34b-afabee439353\" (UID: \"5418c25a-f580-47e0-b34b-afabee439353\") "
Sep 15 11:43:08.239268 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.239272    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjv2b\" (UniqueName: \"kubernetes.io/projected/5418c25a-f580-47e0-b34b-afabee439353-kube-api-access-xjv2b\") pod \"5418c25a-f580-47e0-b34b-afabee439353\" (UID: \"5418c25a-f580-47e0-b34b-afabee439353\") "
Sep 15 11:43:08.239779 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.239732    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5418c25a-f580-47e0-b34b-afabee439353-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5418c25a-f580-47e0-b34b-afabee439353" (UID: "5418c25a-f580-47e0-b34b-afabee439353"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:43:08.241642 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.241614    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5418c25a-f580-47e0-b34b-afabee439353-kube-api-access-xjv2b" (OuterVolumeSpecName: "kube-api-access-xjv2b") pod "5418c25a-f580-47e0-b34b-afabee439353" (UID: "5418c25a-f580-47e0-b34b-afabee439353"). InnerVolumeSpecName "kube-api-access-xjv2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:43:08.340423 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.340391    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5418c25a-f580-47e0-b34b-afabee439353-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:08.340423 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.340421    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjv2b\" (UniqueName: \"kubernetes.io/projected/5418c25a-f580-47e0-b34b-afabee439353-kube-api-access-xjv2b\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:08.391946 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.391909    2560 patch_prober.go:28] interesting pod/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 15 11:43:08.392131 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.391971    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb" podUID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 15 11:43:08.569271 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.569238    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:08.582314 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.582286    2560 generic.go:356] "Generic (PLEG): container finished" podID="5418c25a-f580-47e0-b34b-afabee439353" containerID="b51df9ec34a2a214d228030f5d7373d86d5570c5f74a816187f2f5fb9ca1e5d4" exitCode=0
Sep 15 11:43:08.582461 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.582332    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"
Sep 15 11:43:08.582461 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.582364    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq" event={"ID":"5418c25a-f580-47e0-b34b-afabee439353","Type":"ContainerDied","Data":"b51df9ec34a2a214d228030f5d7373d86d5570c5f74a816187f2f5fb9ca1e5d4"}
Sep 15 11:43:08.582461 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.582417    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq" event={"ID":"5418c25a-f580-47e0-b34b-afabee439353","Type":"ContainerDied","Data":"bfe183050d9a9771d434ea15cabe32aa69e6da44ee675b940b68c910d761a4c8"}
Sep 15 11:43:08.582461 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.582449    2560 scope.go:117] "RemoveContainer" containerID="b51df9ec34a2a214d228030f5d7373d86d5570c5f74a816187f2f5fb9ca1e5d4"
Sep 15 11:43:08.584314 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.584294    2560 generic.go:356] "Generic (PLEG): container finished" podID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" containerID="7a3f3a007c0bae10679b176fa3c70df8d994d50e275995a916b676fe1596e1e0" exitCode=0
Sep 15 11:43:08.584430 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.584367    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb" event={"ID":"8e36e9af-a80b-4e01-b1f6-844ef3040a7f","Type":"ContainerDied","Data":"7a3f3a007c0bae10679b176fa3c70df8d994d50e275995a916b676fe1596e1e0"}
Sep 15 11:43:08.584771 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.584756    2560 scope.go:117] "RemoveContainer" containerID="7a3f3a007c0bae10679b176fa3c70df8d994d50e275995a916b676fe1596e1e0"
Sep 15 11:43:08.585984 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.585953    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:43:08.586095 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.586023    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:43:08.586095 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.586055    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:43:08.592906 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.592889    2560 scope.go:117] "RemoveContainer" containerID="b51df9ec34a2a214d228030f5d7373d86d5570c5f74a816187f2f5fb9ca1e5d4"
Sep 15 11:43:08.593163 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:43:08.593144    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b51df9ec34a2a214d228030f5d7373d86d5570c5f74a816187f2f5fb9ca1e5d4\": container with ID starting with b51df9ec34a2a214d228030f5d7373d86d5570c5f74a816187f2f5fb9ca1e5d4 not found: ID does not exist" containerID="b51df9ec34a2a214d228030f5d7373d86d5570c5f74a816187f2f5fb9ca1e5d4"
Sep 15 11:43:08.593227 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.593172    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b51df9ec34a2a214d228030f5d7373d86d5570c5f74a816187f2f5fb9ca1e5d4"} err="failed to get container status \"b51df9ec34a2a214d228030f5d7373d86d5570c5f74a816187f2f5fb9ca1e5d4\": rpc error: code = NotFound desc = could not find container \"b51df9ec34a2a214d228030f5d7373d86d5570c5f74a816187f2f5fb9ca1e5d4\": container with ID starting with b51df9ec34a2a214d228030f5d7373d86d5570c5f74a816187f2f5fb9ca1e5d4 not found: ID does not exist"
Sep 15 11:43:08.732800 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.732735    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"]
Sep 15 11:43:08.739024 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.738774    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a73e48-7d98954d54-rsmwq"]
Sep 15 11:43:08.782601 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:08.782536    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:09.035817 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:09.035728    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5418c25a-f580-47e0-b34b-afabee439353" path="/var/lib/kubelet/pods/5418c25a-f580-47e0-b34b-afabee439353/volumes"
Sep 15 11:43:09.036257 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:09.036130    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66f17071-39b0-406b-bcf5-462b1b512241" path="/var/lib/kubelet/pods/66f17071-39b0-406b-bcf5-462b1b512241/volumes"
Sep 15 11:43:09.598281 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:09.598237    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb" event={"ID":"8e36e9af-a80b-4e01-b1f6-844ef3040a7f","Type":"ContainerStarted","Data":"38163217ee151c619a61564141d6272c03f214683453b09b28f836331015c08c"}
Sep 15 11:43:09.618494 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:09.618452    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:10.336425 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.336394    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:43:10.336889 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.336631    2560 patch_prober.go:28] interesting pod/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 15 11:43:10.336889 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.336692    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb" podUID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 15 11:43:10.344951 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.344920    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:10.450155 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.450124    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:10.546072 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.546024    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:10.697669 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.697632    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f7ca109-4788v"]
Sep 15 11:43:10.698622 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.698595    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5418c25a-f580-47e0-b34b-afabee439353" containerName="payload-pre-processing"
Sep 15 11:43:10.698783 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.698770    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5418c25a-f580-47e0-b34b-afabee439353" containerName="payload-pre-processing"
Sep 15 11:43:10.698879 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.698869    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f17071-39b0-406b-bcf5-462b1b512241" containerName="revoke-keys"
Sep 15 11:43:10.698960 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.698950    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f17071-39b0-406b-bcf5-462b1b512241" containerName="revoke-keys"
Sep 15 11:43:10.699148 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.699135    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5418c25a-f580-47e0-b34b-afabee439353" containerName="payload-pre-processing"
Sep 15 11:43:10.699247 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.699236    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="66f17071-39b0-406b-bcf5-462b1b512241" containerName="revoke-keys"
Sep 15 11:43:10.704033 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.704011    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f7ca109-4788v"
Sep 15 11:43:10.708590 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.707282    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbxjk\""
Sep 15 11:43:10.708789 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.708738    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f7ca109-4788v"]
Sep 15 11:43:10.762929 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.762896    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae14dfe7-2d42-45cb-9f29-a198cdb24da6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5f7ca109-4788v\" (UID: \"ae14dfe7-2d42-45cb-9f29-a198cdb24da6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f7ca109-4788v"
Sep 15 11:43:10.864498 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.864462    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae14dfe7-2d42-45cb-9f29-a198cdb24da6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5f7ca109-4788v\" (UID: \"ae14dfe7-2d42-45cb-9f29-a198cdb24da6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f7ca109-4788v"
Sep 15 11:43:10.865150 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:10.865126    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae14dfe7-2d42-45cb-9f29-a198cdb24da6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5f7ca109-4788v\" (UID: \"ae14dfe7-2d42-45cb-9f29-a198cdb24da6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f7ca109-4788v"
Sep 15 11:43:11.022751 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:11.022657    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f7ca109-4788v"
Sep 15 11:43:11.304959 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:11.301471    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f7ca109-4788v"]
Sep 15 11:43:11.337121 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:11.337078    2560 patch_prober.go:28] interesting pod/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 15 11:43:11.337501 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:11.337140    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb" podUID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 15 11:43:11.581690 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:11.581595    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:11.618782 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:11.618750    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f7ca109-4788v" event={"ID":"ae14dfe7-2d42-45cb-9f29-a198cdb24da6","Type":"ContainerStarted","Data":"8a5b22e2a60e6ee8b9d62ae4b4b0f8bc5c4d38684e688c622a8ab049f029ebac"}
Sep 15 11:43:12.336644 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:12.336610    2560 patch_prober.go:28] interesting pod/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 15 11:43:12.336821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:12.336664    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb" podUID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 15 11:43:12.410445 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:12.410318    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:12.625443 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:12.625404    2560 generic.go:356] "Generic (PLEG): container finished" podID="ae14dfe7-2d42-45cb-9f29-a198cdb24da6" containerID="26c7af9e516452ea688b4429726b45513f0d9052047623c689c999b8955cfff8" exitCode=0
Sep 15 11:43:12.625635 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:12.625456    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f7ca109-4788v" event={"ID":"ae14dfe7-2d42-45cb-9f29-a198cdb24da6","Type":"ContainerDied","Data":"26c7af9e516452ea688b4429726b45513f0d9052047623c689c999b8955cfff8"}
Sep 15 11:43:13.320187 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:13.320127    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:13.336987 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:13.336955    2560 patch_prober.go:28] interesting pod/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 15 11:43:13.337159 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:13.336999    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb" podUID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 15 11:43:13.782466 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:13.782441    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f7ca109-4788v"
Sep 15 11:43:13.904064 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:13.903963    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae14dfe7-2d42-45cb-9f29-a198cdb24da6-maas-api-service-ca\") pod \"ae14dfe7-2d42-45cb-9f29-a198cdb24da6\" (UID: \"ae14dfe7-2d42-45cb-9f29-a198cdb24da6\") "
Sep 15 11:43:13.905441 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:13.904965    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae14dfe7-2d42-45cb-9f29-a198cdb24da6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ae14dfe7-2d42-45cb-9f29-a198cdb24da6" (UID: "ae14dfe7-2d42-45cb-9f29-a198cdb24da6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:43:14.006877 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:14.006839    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae14dfe7-2d42-45cb-9f29-a198cdb24da6-maas-api-service-ca\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:14.336406 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:14.336326    2560 patch_prober.go:28] interesting pod/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 15 11:43:14.336406 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:14.336378    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb" podUID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 15 11:43:14.638623 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:14.638591    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f7ca109-4788v"
Sep 15 11:43:14.638623 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:14.638604    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f7ca109-4788v" event={"ID":"ae14dfe7-2d42-45cb-9f29-a198cdb24da6","Type":"ContainerDied","Data":"8a5b22e2a60e6ee8b9d62ae4b4b0f8bc5c4d38684e688c622a8ab049f029ebac"}
Sep 15 11:43:14.638884 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:14.638648    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a5b22e2a60e6ee8b9d62ae4b4b0f8bc5c4d38684e688c622a8ab049f029ebac"
Sep 15 11:43:15.055397 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:15.055295    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:15.341176 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:15.341148    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:43:15.341460 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:15.341440    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:43:15.342041 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:15.342023    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:43:15.390703 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:15.390667    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:15.476437 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:15.476402    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:15.644452 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:15.644373    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"]
Sep 15 11:43:15.644731 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:15.644633    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs" podUID="8bd60ba0-bdce-448c-89bf-6ad6dd292a10" containerName="istio-proxy" containerID="cri-o://2c0e02abbfbaae2f60fca18e57da8c0daf3758510243e8774f95c64f5344929b" gracePeriod=30
Sep 15 11:43:15.670539 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:15.670511    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:16.100390 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.100345    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:16.223256 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.223222    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:16.384113 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.384035    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"]
Sep 15 11:43:16.384483 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.384471    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae14dfe7-2d42-45cb-9f29-a198cdb24da6" containerName="revoke-keys"
Sep 15 11:43:16.384533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.384484    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae14dfe7-2d42-45cb-9f29-a198cdb24da6" containerName="revoke-keys"
Sep 15 11:43:16.384606 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.384586    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae14dfe7-2d42-45cb-9f29-a198cdb24da6" containerName="revoke-keys"
Sep 15 11:43:16.387700 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.387674    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.389840 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.389811    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-ed9962-openshift-default-dockercfg-sgxbq\""
Sep 15 11:43:16.400768 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.400746    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"]
Sep 15 11:43:16.421470 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.421444    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:16.533534 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.533497    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-credential-socket\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.533534 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.533532    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-podinfo\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.533824 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.533643    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-workload-socket\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.533824 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.533692    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-envoy\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.533824 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.533719    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6062a779-44ff-4a8e-b63a-5b41580f5913-istiod-ca-cert\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.533824 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.533793    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-data\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.534050 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.533859    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-token\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.534050 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.533907    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfb5m\" (UniqueName: \"kubernetes.io/projected/6062a779-44ff-4a8e-b63a-5b41580f5913-kube-api-access-bfb5m\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.534050 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.533980    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-workload-certs\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.635345 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.635258    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfb5m\" (UniqueName: \"kubernetes.io/projected/6062a779-44ff-4a8e-b63a-5b41580f5913-kube-api-access-bfb5m\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.635345 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.635334    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-workload-certs\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.635596 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.635371    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-credential-socket\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.635596 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.635394    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-podinfo\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.635596 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.635457    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-workload-socket\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.635596 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.635517    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-envoy\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.635596 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.635581    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6062a779-44ff-4a8e-b63a-5b41580f5913-istiod-ca-cert\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.635865 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.635626    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-data\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.635865 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.635664    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-token\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.635865 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.635804    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-workload-certs\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.635865 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.635852    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-credential-socket\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.636045 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.635909    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-workload-socket\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.636083 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.636064    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-data\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.636332 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.636303    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6062a779-44ff-4a8e-b63a-5b41580f5913-istiod-ca-cert\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.637942 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.637916    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-envoy\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.638040 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.638018    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-podinfo\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.642865 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.642840    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-token\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.642865 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.642861    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfb5m\" (UniqueName: \"kubernetes.io/projected/6062a779-44ff-4a8e-b63a-5b41580f5913-kube-api-access-bfb5m\") pod \"e2e-noten-ed9962-openshift-default-87699cbf4-km6hv\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") " pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.699748 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.699715    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:16.771422 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.771165    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:16.983132 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.983042    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"]
Sep 15 11:43:16.987060 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:43:16.987013    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6062a779_44ff_4a8e_b63a_5b41580f5913.slice/crio-be6635f000bad5c7f2ff2d1f9d537e6de0ed85c0c4db8ff71be97763e9cb7bde WatchSource:0}: Error finding container be6635f000bad5c7f2ff2d1f9d537e6de0ed85c0c4db8ff71be97763e9cb7bde: Status 404 returned error can't find the container with id be6635f000bad5c7f2ff2d1f9d537e6de0ed85c0c4db8ff71be97763e9cb7bde
Sep 15 11:43:16.989815 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.989771    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:43:16.989909 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.989874    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:43:16.989956 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:16.989920    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:43:17.658382 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:17.658345    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv" event={"ID":"6062a779-44ff-4a8e-b63a-5b41580f5913","Type":"ContainerStarted","Data":"6bfb112fab99e0c485a16f656148dec36c1ba59aa3172ddb9ac44090514496c6"}
Sep 15 11:43:17.658789 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:17.658406    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv" event={"ID":"6062a779-44ff-4a8e-b63a-5b41580f5913","Type":"ContainerStarted","Data":"be6635f000bad5c7f2ff2d1f9d537e6de0ed85c0c4db8ff71be97763e9cb7bde"}
Sep 15 11:43:17.677362 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:17.677311    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv" podStartSLOduration=1.6772960559999999 podStartE2EDuration="1.677296056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:43:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:43:17.675748453 +0000 UTC m=+1099.168525602" watchObservedRunningTime="2026-09-15 11:43:17.677296056 +0000 UTC m=+1099.170073201"
Sep 15 11:43:17.700043 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:17.700015    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:17.967604 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:17.967489    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:18.061357 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:18.061319    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:18.205060 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:18.205016    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:18.706502 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:18.706469    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:19.669831 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:19.669803    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:19.670941 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:19.670916    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:43:20.913019 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.912960    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:43:20.979683 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.979657    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-podinfo\") pod \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") "
Sep 15 11:43:20.979859 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.979723    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istiod-ca-cert\") pod \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") "
Sep 15 11:43:20.979859 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.979822    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmgvs\" (UniqueName: \"kubernetes.io/projected/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-kube-api-access-pmgvs\") pod \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") "
Sep 15 11:43:20.979859 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.979856    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-envoy\") pod \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") "
Sep 15 11:43:20.980033 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.979877    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-workload-socket\") pod \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") "
Sep 15 11:43:20.980033 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.979959    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-credential-socket\") pod \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") "
Sep 15 11:43:20.980033 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.980010    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-data\") pod \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") "
Sep 15 11:43:20.980196 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.980078    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8bd60ba0-bdce-448c-89bf-6ad6dd292a10" (UID: "8bd60ba0-bdce-448c-89bf-6ad6dd292a10"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:43:20.980196 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.980087    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-token\") pod \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") "
Sep 15 11:43:20.980297 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.980207    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8bd60ba0-bdce-448c-89bf-6ad6dd292a10" (UID: "8bd60ba0-bdce-448c-89bf-6ad6dd292a10"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:43:20.980297 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.980235    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-workload-certs\") pod \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\" (UID: \"8bd60ba0-bdce-448c-89bf-6ad6dd292a10\") "
Sep 15 11:43:20.980413 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.980380    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-data" (OuterVolumeSpecName: "istio-data") pod "8bd60ba0-bdce-448c-89bf-6ad6dd292a10" (UID: "8bd60ba0-bdce-448c-89bf-6ad6dd292a10"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:43:20.980509 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.980477    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8bd60ba0-bdce-448c-89bf-6ad6dd292a10" (UID: "8bd60ba0-bdce-448c-89bf-6ad6dd292a10"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:43:20.980509 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.980494    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8bd60ba0-bdce-448c-89bf-6ad6dd292a10" (UID: "8bd60ba0-bdce-448c-89bf-6ad6dd292a10"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:43:20.980798 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.980777    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:20.980869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.980803    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:20.980869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.980818    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:20.980869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.980830    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:20.980869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.980843    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:20.982019 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.981994    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-kube-api-access-pmgvs" (OuterVolumeSpecName: "kube-api-access-pmgvs") pod "8bd60ba0-bdce-448c-89bf-6ad6dd292a10" (UID: "8bd60ba0-bdce-448c-89bf-6ad6dd292a10"). InnerVolumeSpecName "kube-api-access-pmgvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:43:20.982127 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.982071    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8bd60ba0-bdce-448c-89bf-6ad6dd292a10" (UID: "8bd60ba0-bdce-448c-89bf-6ad6dd292a10"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:43:20.982368 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.982348    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-token" (OuterVolumeSpecName: "istio-token") pod "8bd60ba0-bdce-448c-89bf-6ad6dd292a10" (UID: "8bd60ba0-bdce-448c-89bf-6ad6dd292a10"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:43:20.982446 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:20.982354    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8bd60ba0-bdce-448c-89bf-6ad6dd292a10" (UID: "8bd60ba0-bdce-448c-89bf-6ad6dd292a10"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:43:21.081895 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.081866    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmgvs\" (UniqueName: \"kubernetes.io/projected/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-kube-api-access-pmgvs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:21.081895 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.081891    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:21.081895 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.081900    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:21.082075 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.081907    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bd60ba0-bdce-448c-89bf-6ad6dd292a10-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:21.683043 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.683002    2560 generic.go:356] "Generic (PLEG): container finished" podID="8bd60ba0-bdce-448c-89bf-6ad6dd292a10" containerID="2c0e02abbfbaae2f60fca18e57da8c0daf3758510243e8774f95c64f5344929b" exitCode=0
Sep 15 11:43:21.683253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.683123    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"
Sep 15 11:43:21.683253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.683133    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs" event={"ID":"8bd60ba0-bdce-448c-89bf-6ad6dd292a10","Type":"ContainerDied","Data":"2c0e02abbfbaae2f60fca18e57da8c0daf3758510243e8774f95c64f5344929b"}
Sep 15 11:43:21.683253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.683189    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs" event={"ID":"8bd60ba0-bdce-448c-89bf-6ad6dd292a10","Type":"ContainerDied","Data":"8ee6874474efa4fd2d79a7f4299809759b247cbfd0b4482e9a276fbbb66da855"}
Sep 15 11:43:21.683253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.683211    2560 scope.go:117] "RemoveContainer" containerID="2c0e02abbfbaae2f60fca18e57da8c0daf3758510243e8774f95c64f5344929b"
Sep 15 11:43:21.694654 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.694626    2560 scope.go:117] "RemoveContainer" containerID="651afd7300778c512899d28bd317cdc89b089bd8777a1bc6ca8c41232e32e9f3"
Sep 15 11:43:21.700501 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.700475    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"]
Sep 15 11:43:21.704904 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.704801    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-a73e48-openshift-default-7b6556cdd-gw9vs"]
Sep 15 11:43:21.704964 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.704948    2560 scope.go:117] "RemoveContainer" containerID="2c0e02abbfbaae2f60fca18e57da8c0daf3758510243e8774f95c64f5344929b"
Sep 15 11:43:21.705253 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:43:21.705233    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c0e02abbfbaae2f60fca18e57da8c0daf3758510243e8774f95c64f5344929b\": container with ID starting with 2c0e02abbfbaae2f60fca18e57da8c0daf3758510243e8774f95c64f5344929b not found: ID does not exist" containerID="2c0e02abbfbaae2f60fca18e57da8c0daf3758510243e8774f95c64f5344929b"
Sep 15 11:43:21.705333 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.705265    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c0e02abbfbaae2f60fca18e57da8c0daf3758510243e8774f95c64f5344929b"} err="failed to get container status \"2c0e02abbfbaae2f60fca18e57da8c0daf3758510243e8774f95c64f5344929b\": rpc error: code = NotFound desc = could not find container \"2c0e02abbfbaae2f60fca18e57da8c0daf3758510243e8774f95c64f5344929b\": container with ID starting with 2c0e02abbfbaae2f60fca18e57da8c0daf3758510243e8774f95c64f5344929b not found: ID does not exist"
Sep 15 11:43:21.705333 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.705289    2560 scope.go:117] "RemoveContainer" containerID="651afd7300778c512899d28bd317cdc89b089bd8777a1bc6ca8c41232e32e9f3"
Sep 15 11:43:21.705629 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:43:21.705582    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"651afd7300778c512899d28bd317cdc89b089bd8777a1bc6ca8c41232e32e9f3\": container with ID starting with 651afd7300778c512899d28bd317cdc89b089bd8777a1bc6ca8c41232e32e9f3 not found: ID does not exist" containerID="651afd7300778c512899d28bd317cdc89b089bd8777a1bc6ca8c41232e32e9f3"
Sep 15 11:43:21.705629 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:21.705613    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"651afd7300778c512899d28bd317cdc89b089bd8777a1bc6ca8c41232e32e9f3"} err="failed to get container status \"651afd7300778c512899d28bd317cdc89b089bd8777a1bc6ca8c41232e32e9f3\": rpc error: code = NotFound desc = could not find container \"651afd7300778c512899d28bd317cdc89b089bd8777a1bc6ca8c41232e32e9f3\": container with ID starting with 651afd7300778c512899d28bd317cdc89b089bd8777a1bc6ca8c41232e32e9f3 not found: ID does not exist"
Sep 15 11:43:23.035535 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:23.035503    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bd60ba0-bdce-448c-89bf-6ad6dd292a10" path="/var/lib/kubelet/pods/8bd60ba0-bdce-448c-89bf-6ad6dd292a10/volumes"
Sep 15 11:43:23.588818 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:23.588784    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:24.148429 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:24.148389    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:24.218642 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:24.218600    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:24.318785 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:24.318707    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:24.318974 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:24.318859    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f7ca109-4788v"]
Sep 15 11:43:24.322063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:24.322039    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5f7ca109-4788v"]
Sep 15 11:43:24.410059 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:24.409941    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:24.661088 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:24.660996    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:24.774361 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:24.774327    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:25.035647 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:25.035551    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae14dfe7-2d42-45cb-9f29-a198cdb24da6" path="/var/lib/kubelet/pods/ae14dfe7-2d42-45cb-9f29-a198cdb24da6/volumes"
Sep 15 11:43:25.583231 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:25.583191    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:26.377695 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:26.377661    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:27.256202 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:27.256156    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:27.979443 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:27.979404    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:30.572208 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:30.572171    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:30.672760 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:30.672717    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:30.812320 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:30.812288    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"]
Sep 15 11:43:30.812577 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:30.812532    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb" podUID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" containerName="istio-proxy" containerID="cri-o://38163217ee151c619a61564141d6272c03f214683453b09b28f836331015c08c" gracePeriod=30
Sep 15 11:43:30.818905 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:30.818878    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:31.170333 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:31.170299    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:31.270540 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:31.270501    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:31.369537 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:31.369502    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:34.524344 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:34.524310    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:36.078118 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.078064    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:43:36.144808 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.144775    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4t9w\" (UniqueName: \"kubernetes.io/projected/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-kube-api-access-c4t9w\") pod \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") "
Sep 15 11:43:36.144989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.144826    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-envoy\") pod \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") "
Sep 15 11:43:36.144989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.144862    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-workload-socket\") pod \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") "
Sep 15 11:43:36.144989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.144884    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-workload-certs\") pod \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") "
Sep 15 11:43:36.144989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.144914    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-token\") pod \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") "
Sep 15 11:43:36.144989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.144941    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istiod-ca-cert\") pod \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") "
Sep 15 11:43:36.144989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.144973    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-podinfo\") pod \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") "
Sep 15 11:43:36.145306 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.145030    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-credential-socket\") pod \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") "
Sep 15 11:43:36.145306 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.145075    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-data\") pod \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\" (UID: \"8e36e9af-a80b-4e01-b1f6-844ef3040a7f\") "
Sep 15 11:43:36.145306 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.145232    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8e36e9af-a80b-4e01-b1f6-844ef3040a7f" (UID: "8e36e9af-a80b-4e01-b1f6-844ef3040a7f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:43:36.145452 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.145392    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8e36e9af-a80b-4e01-b1f6-844ef3040a7f" (UID: "8e36e9af-a80b-4e01-b1f6-844ef3040a7f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:43:36.145508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.145477    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8e36e9af-a80b-4e01-b1f6-844ef3040a7f" (UID: "8e36e9af-a80b-4e01-b1f6-844ef3040a7f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:43:36.145508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.145490    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-data" (OuterVolumeSpecName: "istio-data") pod "8e36e9af-a80b-4e01-b1f6-844ef3040a7f" (UID: "8e36e9af-a80b-4e01-b1f6-844ef3040a7f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:43:36.145508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.145500    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:36.145693 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.145520    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:36.145693 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.145671    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8e36e9af-a80b-4e01-b1f6-844ef3040a7f" (UID: "8e36e9af-a80b-4e01-b1f6-844ef3040a7f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:43:36.147086 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.147059    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-kube-api-access-c4t9w" (OuterVolumeSpecName: "kube-api-access-c4t9w") pod "8e36e9af-a80b-4e01-b1f6-844ef3040a7f" (UID: "8e36e9af-a80b-4e01-b1f6-844ef3040a7f"). InnerVolumeSpecName "kube-api-access-c4t9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:43:36.147230 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.147211    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8e36e9af-a80b-4e01-b1f6-844ef3040a7f" (UID: "8e36e9af-a80b-4e01-b1f6-844ef3040a7f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:43:36.147469 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.147446    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8e36e9af-a80b-4e01-b1f6-844ef3040a7f" (UID: "8e36e9af-a80b-4e01-b1f6-844ef3040a7f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:43:36.147600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.147534    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-token" (OuterVolumeSpecName: "istio-token") pod "8e36e9af-a80b-4e01-b1f6-844ef3040a7f" (UID: "8e36e9af-a80b-4e01-b1f6-844ef3040a7f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:43:36.246216 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.246181    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:36.246216 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.246212    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:36.246216 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.246221    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4t9w\" (UniqueName: \"kubernetes.io/projected/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-kube-api-access-c4t9w\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:36.246432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.246231    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:36.246432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.246239    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:36.246432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.246247    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:36.246432 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.246256    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e36e9af-a80b-4e01-b1f6-844ef3040a7f-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:43:36.433102 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.433069    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:36.772961 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.772887    2560 generic.go:356] "Generic (PLEG): container finished" podID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" containerID="38163217ee151c619a61564141d6272c03f214683453b09b28f836331015c08c" exitCode=0
Sep 15 11:43:36.772961 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.772930    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"
Sep 15 11:43:36.773251 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.772964    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb" event={"ID":"8e36e9af-a80b-4e01-b1f6-844ef3040a7f","Type":"ContainerDied","Data":"38163217ee151c619a61564141d6272c03f214683453b09b28f836331015c08c"}
Sep 15 11:43:36.773251 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.773037    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb" event={"ID":"8e36e9af-a80b-4e01-b1f6-844ef3040a7f","Type":"ContainerDied","Data":"c84a862ef66f35bdde091d269610c11b200ada0fd12f61a03946ae79526e35e7"}
Sep 15 11:43:36.773251 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.773058    2560 scope.go:117] "RemoveContainer" containerID="38163217ee151c619a61564141d6272c03f214683453b09b28f836331015c08c"
Sep 15 11:43:36.783455 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.783439    2560 scope.go:117] "RemoveContainer" containerID="7a3f3a007c0bae10679b176fa3c70df8d994d50e275995a916b676fe1596e1e0"
Sep 15 11:43:36.793704 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.793688    2560 scope.go:117] "RemoveContainer" containerID="38163217ee151c619a61564141d6272c03f214683453b09b28f836331015c08c"
Sep 15 11:43:36.794009 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:43:36.793985    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38163217ee151c619a61564141d6272c03f214683453b09b28f836331015c08c\": container with ID starting with 38163217ee151c619a61564141d6272c03f214683453b09b28f836331015c08c not found: ID does not exist" containerID="38163217ee151c619a61564141d6272c03f214683453b09b28f836331015c08c"
Sep 15 11:43:36.794098 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.794020    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38163217ee151c619a61564141d6272c03f214683453b09b28f836331015c08c"} err="failed to get container status \"38163217ee151c619a61564141d6272c03f214683453b09b28f836331015c08c\": rpc error: code = NotFound desc = could not find container \"38163217ee151c619a61564141d6272c03f214683453b09b28f836331015c08c\": container with ID starting with 38163217ee151c619a61564141d6272c03f214683453b09b28f836331015c08c not found: ID does not exist"
Sep 15 11:43:36.794098 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.794043    2560 scope.go:117] "RemoveContainer" containerID="7a3f3a007c0bae10679b176fa3c70df8d994d50e275995a916b676fe1596e1e0"
Sep 15 11:43:36.794314 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:43:36.794293    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a3f3a007c0bae10679b176fa3c70df8d994d50e275995a916b676fe1596e1e0\": container with ID starting with 7a3f3a007c0bae10679b176fa3c70df8d994d50e275995a916b676fe1596e1e0 not found: ID does not exist" containerID="7a3f3a007c0bae10679b176fa3c70df8d994d50e275995a916b676fe1596e1e0"
Sep 15 11:43:36.794382 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.794321    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a3f3a007c0bae10679b176fa3c70df8d994d50e275995a916b676fe1596e1e0"} err="failed to get container status \"7a3f3a007c0bae10679b176fa3c70df8d994d50e275995a916b676fe1596e1e0\": rpc error: code = NotFound desc = could not find container \"7a3f3a007c0bae10679b176fa3c70df8d994d50e275995a916b676fe1596e1e0\": container with ID starting with 7a3f3a007c0bae10679b176fa3c70df8d994d50e275995a916b676fe1596e1e0 not found: ID does not exist"
Sep 15 11:43:36.795981 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.795933    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"]
Sep 15 11:43:36.799702 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:36.799682    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5f7ca109-openshift-default-557bb948fb-657tb"]
Sep 15 11:43:37.035366 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:37.035274    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" path="/var/lib/kubelet/pods/8e36e9af-a80b-4e01-b1f6-844ef3040a7f/volumes"
Sep 15 11:43:37.252973 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:37.252938    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:37.884804 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:37.884776    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:37.988537 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:37.988503    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:38.089681 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:38.089648    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:38.211274 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:38.211184    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:44.181001 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:44.180965    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:44.250089 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:44.250037    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:45.059111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.059078    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:45.329439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.329359    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"]
Sep 15 11:43:45.329820 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.329805    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bd60ba0-bdce-448c-89bf-6ad6dd292a10" containerName="istio-proxy"
Sep 15 11:43:45.329891 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.329821    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bd60ba0-bdce-448c-89bf-6ad6dd292a10" containerName="istio-proxy"
Sep 15 11:43:45.329891 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.329844    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" containerName="istio-proxy"
Sep 15 11:43:45.329891 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.329850    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" containerName="istio-proxy"
Sep 15 11:43:45.329891 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.329863    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" containerName="istio-proxy"
Sep 15 11:43:45.329891 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.329868    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" containerName="istio-proxy"
Sep 15 11:43:45.330219 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.329940    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bd60ba0-bdce-448c-89bf-6ad6dd292a10" containerName="istio-proxy"
Sep 15 11:43:45.330219 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.329951    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" containerName="istio-proxy"
Sep 15 11:43:45.333415 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.333397    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.335471 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.335453    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-5a8d6f-openshift-default-dockercfg-twjtm\""
Sep 15 11:43:45.347927 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.347902    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"]
Sep 15 11:43:45.369631 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.369597    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:45.433031 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.432944    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-istio-data\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.433224 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.433082    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-workload-certs\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.433224 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.433186    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af3d83d7-8709-4154-976f-695994a8cebb-istiod-ca-cert\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.433224 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.433214    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-credential-socket\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.433433 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.433241    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af3d83d7-8709-4154-976f-695994a8cebb-istio-token\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.433433 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.433281    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdn2s\" (UniqueName: \"kubernetes.io/projected/af3d83d7-8709-4154-976f-695994a8cebb-kube-api-access-qdn2s\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.433433 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.433337    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-workload-socket\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.433433 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.433376    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-istio-envoy\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.433433 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.433427    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af3d83d7-8709-4154-976f-695994a8cebb-istio-podinfo\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.534317 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.534285    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af3d83d7-8709-4154-976f-695994a8cebb-istiod-ca-cert\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.534317 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.534319    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-credential-socket\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.534538 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.534340    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af3d83d7-8709-4154-976f-695994a8cebb-istio-token\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.534538 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.534371    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdn2s\" (UniqueName: \"kubernetes.io/projected/af3d83d7-8709-4154-976f-695994a8cebb-kube-api-access-qdn2s\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.534538 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.534399    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-workload-socket\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.534538 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.534423    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-istio-envoy\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.534538 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.534468    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af3d83d7-8709-4154-976f-695994a8cebb-istio-podinfo\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.534538 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.534514    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-istio-data\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.534785 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.534590    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-workload-certs\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.535005 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.534974    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-credential-socket\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.535113 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.535044    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-workload-socket\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.535113 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.535091    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-workload-certs\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.535231 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.535160    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-istio-data\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.535290 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.535273    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af3d83d7-8709-4154-976f-695994a8cebb-istiod-ca-cert\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.536928 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.536905    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-istio-envoy\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.537139 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.537120    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af3d83d7-8709-4154-976f-695994a8cebb-istio-podinfo\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.542056 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.542032    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af3d83d7-8709-4154-976f-695994a8cebb-istio-token\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.542287 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.542268    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdn2s\" (UniqueName: \"kubernetes.io/projected/af3d83d7-8709-4154-976f-695994a8cebb-kube-api-access-qdn2s\") pod \"e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") " pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.644319 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.644286    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:45.924442 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.924416    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"]
Sep 15 11:43:45.926429 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:43:45.926397    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf3d83d7_8709_4154_976f_695994a8cebb.slice/crio-1d1f2b312b28215db2fa7b0ba64a50a19f655ec0744cc9c1781169edd730b686 WatchSource:0}: Error finding container 1d1f2b312b28215db2fa7b0ba64a50a19f655ec0744cc9c1781169edd730b686: Status 404 returned error can't find the container with id 1d1f2b312b28215db2fa7b0ba64a50a19f655ec0744cc9c1781169edd730b686
Sep 15 11:43:45.928771 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.928733    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:43:45.928870 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.928799    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:43:45.928870 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:45.928830    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:43:46.093173 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:46.093136    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:46.838224 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:46.838190    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw" event={"ID":"af3d83d7-8709-4154-976f-695994a8cebb","Type":"ContainerStarted","Data":"af80e6eb002febdb0a92e37fc026fb323a7ee73ac331095d43a85b06c641ed12"}
Sep 15 11:43:46.838702 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:46.838267    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw" event={"ID":"af3d83d7-8709-4154-976f-695994a8cebb","Type":"ContainerStarted","Data":"1d1f2b312b28215db2fa7b0ba64a50a19f655ec0744cc9c1781169edd730b686"}
Sep 15 11:43:46.859942 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:46.859891    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw" podStartSLOduration=1.859877391 podStartE2EDuration="1.859877391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:43:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:43:46.858942712 +0000 UTC m=+1128.351719880" watchObservedRunningTime="2026-09-15 11:43:46.859877391 +0000 UTC m=+1128.352654538"
Sep 15 11:43:47.645071 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:47.645035    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:47.649895 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:47.649872    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:47.843972 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:47.843943    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:47.844992 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:47.844969    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:43:48.545080 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:48.545031    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:48.821490 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:48.821372    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:48.905952 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:48.905898    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:49.583809 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:49.583771    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:50.475165 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:50.475130    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:51.274873 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:51.274815    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:52.052302 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:52.052266    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:52.153408 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:52.153361    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:52.409072 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:52.409036    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:52.512190 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:52.512152    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:52.597729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:52.597695    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:54.016762 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:54.016698    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:56.727825 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:56.727791    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:43:56.828157 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:43:56.828122    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:02.804660 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:02.804609    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:02.904175 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:02.904138    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:10.979657 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:10.979539    2560 generic.go:356] "Generic (PLEG): container finished" podID="af3d83d7-8709-4154-976f-695994a8cebb" containerID="af80e6eb002febdb0a92e37fc026fb323a7ee73ac331095d43a85b06c641ed12" exitCode=0
Sep 15 11:44:10.979657 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:10.979609    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw" event={"ID":"af3d83d7-8709-4154-976f-695994a8cebb","Type":"ContainerDied","Data":"af80e6eb002febdb0a92e37fc026fb323a7ee73ac331095d43a85b06c641ed12"}
Sep 15 11:44:10.980098 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:10.980069    2560 scope.go:117] "RemoveContainer" containerID="af80e6eb002febdb0a92e37fc026fb323a7ee73ac331095d43a85b06c641ed12"
Sep 15 11:44:10.981403 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:10.981373    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:44:10.981466 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:10.981442    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:44:10.981505 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:10.981472    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:44:11.987752 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:11.986951    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw" event={"ID":"af3d83d7-8709-4154-976f-695994a8cebb","Type":"ContainerStarted","Data":"0fb28d26848e4a5e5e38eb4e8e22a3fb0fccc2a2bda2c0b0e89869382933f19f"}
Sep 15 11:44:12.645329 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:12.645296    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:44:12.645506 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:12.645470    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 15 11:44:12.645550 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:12.645529    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw" podUID="af3d83d7-8709-4154-976f-695994a8cebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 15 11:44:13.102245 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:13.102193    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:13.645531 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:13.645497    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 15 11:44:13.645727 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:13.645552    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw" podUID="af3d83d7-8709-4154-976f-695994a8cebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 15 11:44:14.645520 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:14.645483    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 15 11:44:14.645991 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:14.645540    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw" podUID="af3d83d7-8709-4154-976f-695994a8cebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 15 11:44:14.705916 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:14.705881    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:15.619678 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:15.619623    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:15.644824 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:15.644782    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 15 11:44:15.644981 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:15.644851    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw" podUID="af3d83d7-8709-4154-976f-695994a8cebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 15 11:44:16.544928 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:16.544895    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:16.644844 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:16.644812    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Sep 15 11:44:16.645019 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:16.644863    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw" podUID="af3d83d7-8709-4154-976f-695994a8cebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Sep 15 11:44:17.315128 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:17.315092    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:17.649206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:17.649173    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:44:17.649820 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:17.649581    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:44:17.650248 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:17.650224    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:44:18.007278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:18.007181    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:18.788057 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:18.788020    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:18.891552 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:18.891517    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:19.156050 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.156020    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:19.413266 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.413190    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"]
Sep 15 11:44:19.413743 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.413728    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bd60ba0-bdce-448c-89bf-6ad6dd292a10" containerName="istio-proxy"
Sep 15 11:44:19.413807 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.413746    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bd60ba0-bdce-448c-89bf-6ad6dd292a10" containerName="istio-proxy"
Sep 15 11:44:19.413856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.413842    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bd60ba0-bdce-448c-89bf-6ad6dd292a10" containerName="istio-proxy"
Sep 15 11:44:19.413856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.413855    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e36e9af-a80b-4e01-b1f6-844ef3040a7f" containerName="istio-proxy"
Sep 15 11:44:19.418056 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.418034    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.420399 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.420377    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-d63075-openshift-default-dockercfg-8t6ds\""
Sep 15 11:44:19.440198 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.440159    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"]
Sep 15 11:44:19.451022 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.450985    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:19.482046 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.482016    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-workload-certs\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.482174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.482058    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-workload-socket\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.482174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.482073    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-data\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.482174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.482088    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istiod-ca-cert\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.482174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.482132    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-credential-socket\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.482377 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.482255    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-token\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.482377 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.482285    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t554r\" (UniqueName: \"kubernetes.io/projected/371c85ad-9ab7-4fad-b7be-f13b7d859a43-kube-api-access-t554r\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.482486 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.482399    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-envoy\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.482486 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.482432    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-podinfo\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.583965 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.583933    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-credential-socket\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.584117 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.584011    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-token\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.584117 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.584040    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t554r\" (UniqueName: \"kubernetes.io/projected/371c85ad-9ab7-4fad-b7be-f13b7d859a43-kube-api-access-t554r\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.584241 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.584116    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-envoy\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.584241 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.584140    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-podinfo\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.584241 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.584189    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-workload-certs\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.584241 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.584229    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-workload-socket\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.584453 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.584253    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-data\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.584453 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.584277    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istiod-ca-cert\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.584453 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.584400    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-credential-socket\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.584643 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.584620    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-workload-certs\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.584698 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.584651    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-data\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.584757 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.584706    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-workload-socket\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.584930 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.584911    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istiod-ca-cert\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.586465 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.586436    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-envoy\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.586703 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.586672    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-podinfo\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.592472 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.592446    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-token\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.592623 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.592528    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t554r\" (UniqueName: \"kubernetes.io/projected/371c85ad-9ab7-4fad-b7be-f13b7d859a43-kube-api-access-t554r\") pod \"e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") " pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:19.702123 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.702037    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:19.731988 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:19.731964    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:20.020169 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:20.020144    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"]
Sep 15 11:44:20.022405 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:44:20.022378    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod371c85ad_9ab7_4fad_b7be_f13b7d859a43.slice/crio-31f28fc697319364ea063f0f0d2cf1347d552a2cfd509270846bcf656dad97ab WatchSource:0}: Error finding container 31f28fc697319364ea063f0f0d2cf1347d552a2cfd509270846bcf656dad97ab: Status 404 returned error can't find the container with id 31f28fc697319364ea063f0f0d2cf1347d552a2cfd509270846bcf656dad97ab
Sep 15 11:44:20.025163 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:20.025130    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:44:20.025233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:20.025199    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:44:20.025233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:20.025222    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:44:20.044787 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:20.044758    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" event={"ID":"371c85ad-9ab7-4fad-b7be-f13b7d859a43","Type":"ContainerStarted","Data":"31f28fc697319364ea063f0f0d2cf1347d552a2cfd509270846bcf656dad97ab"}
Sep 15 11:44:21.050943 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:21.050904    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" event={"ID":"371c85ad-9ab7-4fad-b7be-f13b7d859a43","Type":"ContainerStarted","Data":"6d303c4f4806f992d3a5a09af6480ec0c887a328b661dbf4a42d3a08eff8a431"}
Sep 15 11:44:21.074079 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:21.074030    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" podStartSLOduration=2.074015638 podStartE2EDuration="2.074015638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:44:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:44:21.071891565 +0000 UTC m=+1162.564668806" watchObservedRunningTime="2026-09-15 11:44:21.074015638 +0000 UTC m=+1162.566792786"
Sep 15 11:44:21.732466 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:21.732435    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:21.737774 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:21.737752    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:22.056751 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:22.056662    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:22.057684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:22.057660    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:22.279721 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:22.279686    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:22.975508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:22.975460    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:23.076504 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:23.076454    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:24.068168 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:24.068140    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:24.846707 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:24.846675    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:25.507909 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:25.507873    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:26.264279 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:26.264245    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:26.969275 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:26.969240    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:27.064506 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:27.064464    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:27.216028 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:27.215990    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:27.617828 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:27.617790    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:27.707310 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:27.707277    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"]
Sep 15 11:44:27.709360 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:27.709332    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:27.711174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:27.711157    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"
Sep 15 11:44:27.713272 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:27.713251    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-d63075\""
Sep 15 11:44:27.722328 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:27.722305    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"]
Sep 15 11:44:27.776642 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:27.776604    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsxl7\" (UniqueName: \"kubernetes.io/projected/24bb9262-01ea-495e-a22c-5c60c0ea27d5-kube-api-access-bsxl7\") pod \"payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr\" (UID: \"24bb9262-01ea-495e-a22c-5c60c0ea27d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"
Sep 15 11:44:27.776816 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:27.776647    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24bb9262-01ea-495e-a22c-5c60c0ea27d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr\" (UID: \"24bb9262-01ea-495e-a22c-5c60c0ea27d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"
Sep 15 11:44:27.879810 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:27.879232    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24bb9262-01ea-495e-a22c-5c60c0ea27d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr\" (UID: \"24bb9262-01ea-495e-a22c-5c60c0ea27d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"
Sep 15 11:44:27.879810 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:27.879463    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsxl7\" (UniqueName: \"kubernetes.io/projected/24bb9262-01ea-495e-a22c-5c60c0ea27d5-kube-api-access-bsxl7\") pod \"payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr\" (UID: \"24bb9262-01ea-495e-a22c-5c60c0ea27d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"
Sep 15 11:44:27.880523 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:27.880495    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24bb9262-01ea-495e-a22c-5c60c0ea27d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr\" (UID: \"24bb9262-01ea-495e-a22c-5c60c0ea27d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"
Sep 15 11:44:27.888606 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:27.888579    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsxl7\" (UniqueName: \"kubernetes.io/projected/24bb9262-01ea-495e-a22c-5c60c0ea27d5-kube-api-access-bsxl7\") pod \"payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr\" (UID: \"24bb9262-01ea-495e-a22c-5c60c0ea27d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"
Sep 15 11:44:28.023433 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:28.023402    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"
Sep 15 11:44:28.298374 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:28.298348    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"]
Sep 15 11:44:28.300184 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:44:28.300153    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24bb9262_01ea_495e_a22c_5c60c0ea27d5.slice/crio-305697e4e56b4db3070fe0f7462b8d0b3b113e04e4682965678e1d5e8fcf4723 WatchSource:0}: Error finding container 305697e4e56b4db3070fe0f7462b8d0b3b113e04e4682965678e1d5e8fcf4723: Status 404 returned error can't find the container with id 305697e4e56b4db3070fe0f7462b8d0b3b113e04e4682965678e1d5e8fcf4723
Sep 15 11:44:29.111988 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:29.111954    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr" event={"ID":"24bb9262-01ea-495e-a22c-5c60c0ea27d5","Type":"ContainerStarted","Data":"d67a8d724ccb28a9f8ad971494877fcc65509d26c7d99d89eb2402a28e4e15f0"}
Sep 15 11:44:29.112466 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:29.112019    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"
Sep 15 11:44:29.112466 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:29.112047    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr" event={"ID":"24bb9262-01ea-495e-a22c-5c60c0ea27d5","Type":"ContainerStarted","Data":"305697e4e56b4db3070fe0f7462b8d0b3b113e04e4682965678e1d5e8fcf4723"}
Sep 15 11:44:29.132207 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:29.132156    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr" podStartSLOduration=2.132140791 podStartE2EDuration="2.132140791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:44:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:44:29.130106008 +0000 UTC m=+1170.622883157" watchObservedRunningTime="2026-09-15 11:44:29.132140791 +0000 UTC m=+1170.624917938"
Sep 15 11:44:40.119143 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:40.119110    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"
Sep 15 11:44:40.575607 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:40.575505    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:44.395319 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:44.395266    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:48.188524 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:48.188463    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:49.488875 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:49.488837    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:50.174632 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:50.174553    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:50.981064 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:50.981028    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:51.679668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:51.679555    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:52.058368 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:52.058278    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 15 11:44:52.058368 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:52.058352    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 15 11:44:52.261475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:52.261441    2560 generic.go:356] "Generic (PLEG): container finished" podID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerID="6d303c4f4806f992d3a5a09af6480ec0c887a328b661dbf4a42d3a08eff8a431" exitCode=0
Sep 15 11:44:52.261699 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:52.261514    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" event={"ID":"371c85ad-9ab7-4fad-b7be-f13b7d859a43","Type":"ContainerDied","Data":"6d303c4f4806f992d3a5a09af6480ec0c887a328b661dbf4a42d3a08eff8a431"}
Sep 15 11:44:52.261958 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:52.261939    2560 scope.go:117] "RemoveContainer" containerID="6d303c4f4806f992d3a5a09af6480ec0c887a328b661dbf4a42d3a08eff8a431"
Sep 15 11:44:52.263177 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:52.263149    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:44:52.263232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:52.263216    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:44:52.263272 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:52.263247    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:44:52.388112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:52.388068    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:52.957532 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:52.957502    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:53.269770 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:53.269678    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" event={"ID":"371c85ad-9ab7-4fad-b7be-f13b7d859a43","Type":"ContainerStarted","Data":"454c2f4fc36b0d0ed83ca04c3315bb8840603409f80fe51fbb02909e980ad6b1"}
Sep 15 11:44:53.296373 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:53.296337    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:53.479554 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:53.479511    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:53.732239 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:53.732141    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:53.732489 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:53.732451    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 15 11:44:53.732628 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:53.732515    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 15 11:44:54.127278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:54.127246    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:44:54.732786 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:54.732755    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 15 11:44:54.733166 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:54.732808    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 15 11:44:55.732613 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:55.732554    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 15 11:44:55.732780 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:55.732643    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 15 11:44:56.732316 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:56.732275    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 15 11:44:56.732729 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:56.732344    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 15 11:44:57.733166 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:57.733125    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 15 11:44:57.733660 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:57.733198    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 15 11:44:58.733362 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:58.733320    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 15 11:44:58.733771 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:58.733388    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 15 11:44:59.600305 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:59.600275    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:44:59.601651 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:59.601628    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:44:59.736477 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:59.736446    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:59.736972 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:59.736750    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:44:59.737577 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:44:59.737541    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:45:00.149397 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.149367    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d63075-29824545-lwlw8"]
Sep 15 11:45:00.153122 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.153105    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29824545-jjjxz"]
Sep 15 11:45:00.153232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.153208    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d63075-29824545-lwlw8"
Sep 15 11:45:00.156572 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.156547    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824545-jjjxz"
Sep 15 11:45:00.156661 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.156555    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbxjk\""
Sep 15 11:45:00.164397 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.164380    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d63075-29824545-lwlw8"
Sep 15 11:45:00.170887 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.170854    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824545-jjjxz"
Sep 15 11:45:00.171803 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.171745    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d63075-29824545-lwlw8"]
Sep 15 11:45:00.174274 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.174252    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29824545-jjjxz"]
Sep 15 11:45:00.187533 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.187498    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b8fbdd-29824545-bmjws"]
Sep 15 11:45:00.191796 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.191779    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b8fbdd-29824545-bmjws"
Sep 15 11:45:00.204214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.204152    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b8fbdd-29824545-bmjws"]
Sep 15 11:45:00.226369 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.226288    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b8fbdd-29824545-bmjws"
Sep 15 11:45:00.470207 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.468658    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d63075-29824545-lwlw8"]
Sep 15 11:45:00.474155 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.474132    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29824545-jjjxz"]
Sep 15 11:45:00.475943 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:45:00.475918    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc106311e_7d48_411e_8eb2_7e539449d9a3.slice/crio-739cf2a4f1f8adcde53cca5ffd34c3a15e13b3cc0a4c4794ea8f0e686769910d WatchSource:0}: Error finding container 739cf2a4f1f8adcde53cca5ffd34c3a15e13b3cc0a4c4794ea8f0e686769910d: Status 404 returned error can't find the container with id 739cf2a4f1f8adcde53cca5ffd34c3a15e13b3cc0a4c4794ea8f0e686769910d
Sep 15 11:45:00.476857 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:45:00.476832    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f3ab4b_a83f_43e2_b96e_d90b3dab9a3d.slice/crio-fadae8bf0317c51989bee589d4b5c462e77f140365240ebd75f9927821f1bfbf WatchSource:0}: Error finding container fadae8bf0317c51989bee589d4b5c462e77f140365240ebd75f9927821f1bfbf: Status 404 returned error can't find the container with id fadae8bf0317c51989bee589d4b5c462e77f140365240ebd75f9927821f1bfbf
Sep 15 11:45:00.520464 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:00.520438    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b8fbdd-29824545-bmjws"]
Sep 15 11:45:00.523319 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:45:00.523298    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e14abc_b9da_426e_9f2c_cdfc66c59637.slice/crio-f7f8ed809aa0195eb7228a966aefba710115b18e895a5b9c7acceeeb948d90ae WatchSource:0}: Error finding container f7f8ed809aa0195eb7228a966aefba710115b18e895a5b9c7acceeeb948d90ae: Status 404 returned error can't find the container with id f7f8ed809aa0195eb7228a966aefba710115b18e895a5b9c7acceeeb948d90ae
Sep 15 11:45:01.323035 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:01.323001    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824545-jjjxz" event={"ID":"68f3ab4b-a83f-43e2-b96e-d90b3dab9a3d","Type":"ContainerStarted","Data":"7790c6e675cf7e64af0bc971135f914002e7cdcbc673b252253b2024570e5272"}
Sep 15 11:45:01.323467 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:01.323054    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824545-jjjxz" event={"ID":"68f3ab4b-a83f-43e2-b96e-d90b3dab9a3d","Type":"ContainerStarted","Data":"fadae8bf0317c51989bee589d4b5c462e77f140365240ebd75f9927821f1bfbf"}
Sep 15 11:45:01.324633 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:01.324601    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b8fbdd-29824545-bmjws" event={"ID":"b4e14abc-b9da-426e-9f2c-cdfc66c59637","Type":"ContainerStarted","Data":"8050f257031b2114f28d3332fa0ccd6fe3ecbe6b0b1b5bae5f2fc34ef1cf61d5"}
Sep 15 11:45:01.324776 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:01.324645    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b8fbdd-29824545-bmjws" event={"ID":"b4e14abc-b9da-426e-9f2c-cdfc66c59637","Type":"ContainerStarted","Data":"f7f8ed809aa0195eb7228a966aefba710115b18e895a5b9c7acceeeb948d90ae"}
Sep 15 11:45:01.326235 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:01.326212    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d63075-29824545-lwlw8" event={"ID":"c106311e-7d48-411e-8eb2-7e539449d9a3","Type":"ContainerStarted","Data":"5152b7c5a96a3304bd1749f59af40fa4091ab226677b6938836e17365cb0210c"}
Sep 15 11:45:01.326317 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:01.326257    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d63075-29824545-lwlw8" event={"ID":"c106311e-7d48-411e-8eb2-7e539449d9a3","Type":"ContainerStarted","Data":"739cf2a4f1f8adcde53cca5ffd34c3a15e13b3cc0a4c4794ea8f0e686769910d"}
Sep 15 11:45:01.339321 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:01.339272    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824545-jjjxz" podStartSLOduration=1.339254751 podStartE2EDuration="1.339254751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:45:01.336147102 +0000 UTC m=+1202.828924263" watchObservedRunningTime="2026-09-15 11:45:01.339254751 +0000 UTC m=+1202.832031898"
Sep 15 11:45:01.351647 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:01.351607    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d63075-29824545-lwlw8" podStartSLOduration=1.351593571 podStartE2EDuration="1.351593571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:45:01.349972455 +0000 UTC m=+1202.842749598" watchObservedRunningTime="2026-09-15 11:45:01.351593571 +0000 UTC m=+1202.844370721"
Sep 15 11:45:01.364547 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:01.364505    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b8fbdd-29824545-bmjws" podStartSLOduration=1.364488832 podStartE2EDuration="1.364488832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:45:01.364292533 +0000 UTC m=+1202.857069694" watchObservedRunningTime="2026-09-15 11:45:01.364488832 +0000 UTC m=+1202.857266004"
Sep 15 11:45:02.333672 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:02.333634    2560 generic.go:356] "Generic (PLEG): container finished" podID="b4e14abc-b9da-426e-9f2c-cdfc66c59637" containerID="8050f257031b2114f28d3332fa0ccd6fe3ecbe6b0b1b5bae5f2fc34ef1cf61d5" exitCode=0
Sep 15 11:45:02.334154 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:02.333710    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b8fbdd-29824545-bmjws" event={"ID":"b4e14abc-b9da-426e-9f2c-cdfc66c59637","Type":"ContainerDied","Data":"8050f257031b2114f28d3332fa0ccd6fe3ecbe6b0b1b5bae5f2fc34ef1cf61d5"}
Sep 15 11:45:02.335319 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:02.335291    2560 generic.go:356] "Generic (PLEG): container finished" podID="c106311e-7d48-411e-8eb2-7e539449d9a3" containerID="5152b7c5a96a3304bd1749f59af40fa4091ab226677b6938836e17365cb0210c" exitCode=0
Sep 15 11:45:02.335421 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:02.335366    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d63075-29824545-lwlw8" event={"ID":"c106311e-7d48-411e-8eb2-7e539449d9a3","Type":"ContainerDied","Data":"5152b7c5a96a3304bd1749f59af40fa4091ab226677b6938836e17365cb0210c"}
Sep 15 11:45:02.336863 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:02.336845    2560 generic.go:356] "Generic (PLEG): container finished" podID="68f3ab4b-a83f-43e2-b96e-d90b3dab9a3d" containerID="7790c6e675cf7e64af0bc971135f914002e7cdcbc673b252253b2024570e5272" exitCode=0
Sep 15 11:45:02.336938 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:02.336920    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824545-jjjxz" event={"ID":"68f3ab4b-a83f-43e2-b96e-d90b3dab9a3d","Type":"ContainerDied","Data":"7790c6e675cf7e64af0bc971135f914002e7cdcbc673b252253b2024570e5272"}
Sep 15 11:45:03.455960 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.455921    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:03.491977 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.491946    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824545-jjjxz"
Sep 15 11:45:03.536474 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.536450    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b8fbdd-29824545-bmjws"
Sep 15 11:45:03.571332 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.571309    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d63075-29824545-lwlw8"
Sep 15 11:45:03.715129 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.715053    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"]
Sep 15 11:45:03.715717 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.715691    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c106311e-7d48-411e-8eb2-7e539449d9a3" containerName="cleanup"
Sep 15 11:45:03.715717 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.715717    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c106311e-7d48-411e-8eb2-7e539449d9a3" containerName="cleanup"
Sep 15 11:45:03.715918 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.715731    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68f3ab4b-a83f-43e2-b96e-d90b3dab9a3d" containerName="cleanup"
Sep 15 11:45:03.715918 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.715739    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="68f3ab4b-a83f-43e2-b96e-d90b3dab9a3d" containerName="cleanup"
Sep 15 11:45:03.715918 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.715774    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4e14abc-b9da-426e-9f2c-cdfc66c59637" containerName="cleanup"
Sep 15 11:45:03.715918 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.715783    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4e14abc-b9da-426e-9f2c-cdfc66c59637" containerName="cleanup"
Sep 15 11:45:03.715918 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.715915    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="68f3ab4b-a83f-43e2-b96e-d90b3dab9a3d" containerName="cleanup"
Sep 15 11:45:03.716195 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.715929    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4e14abc-b9da-426e-9f2c-cdfc66c59637" containerName="cleanup"
Sep 15 11:45:03.716195 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.715940    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c106311e-7d48-411e-8eb2-7e539449d9a3" containerName="cleanup"
Sep 15 11:45:03.719067 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.719048    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.721234 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.721215    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-baee2b93-openshift-default-dockercfg-2rlk9\""
Sep 15 11:45:03.731167 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.731142    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"]
Sep 15 11:45:03.740375 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.740350    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:03.852381 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.852337    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-workload-certs\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.852591 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.852416    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a22688e9-f79b-469d-bcb9-8ea2edc86943-istiod-ca-cert\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.852591 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.852462    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-podinfo\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.852591 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.852520    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-credential-socket\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.852591 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.852572    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwt6s\" (UniqueName: \"kubernetes.io/projected/a22688e9-f79b-469d-bcb9-8ea2edc86943-kube-api-access-jwt6s\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.852853 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.852669    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-data\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.852853 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.852715    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-workload-socket\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.852853 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.852744    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-envoy\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.852853 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.852775    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-token\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.943190 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.943154    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:03.954240 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.954214    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-data\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.954345 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.954254    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-workload-socket\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.954345 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.954282    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-envoy\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.954459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.954403    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-token\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.954514 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.954457    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-workload-certs\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.954600 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.954525    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a22688e9-f79b-469d-bcb9-8ea2edc86943-istiod-ca-cert\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.954667 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.954610    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-podinfo\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.954728 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.954674    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-credential-socket\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.954728 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.954715    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwt6s\" (UniqueName: \"kubernetes.io/projected/a22688e9-f79b-469d-bcb9-8ea2edc86943-kube-api-access-jwt6s\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.954832 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.954756    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-data\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.954888 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.954821    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-workload-certs\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.954961 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.954924    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-workload-socket\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.955093 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.955074    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-credential-socket\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.955380 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.955356    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a22688e9-f79b-469d-bcb9-8ea2edc86943-istiod-ca-cert\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.956661 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.956638    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-envoy\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.956882 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.956863    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-podinfo\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.963208 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.963188    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwt6s\" (UniqueName: \"kubernetes.io/projected/a22688e9-f79b-469d-bcb9-8ea2edc86943-kube-api-access-jwt6s\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:03.965966 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:03.965922    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-token\") pod \"e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") " pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:04.029840 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:04.029800    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:04.308426 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:04.308397    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"]
Sep 15 11:45:04.309809 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:45:04.309783    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda22688e9_f79b_469d_bcb9_8ea2edc86943.slice/crio-7f15011fb0433456edbdf35115276f83fde724fb69f42d2ce3f26d485d3589ad WatchSource:0}: Error finding container 7f15011fb0433456edbdf35115276f83fde724fb69f42d2ce3f26d485d3589ad: Status 404 returned error can't find the container with id 7f15011fb0433456edbdf35115276f83fde724fb69f42d2ce3f26d485d3589ad
Sep 15 11:45:04.312354 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:04.312320    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:45:04.312450 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:04.312386    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:45:04.312450 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:04.312416    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:45:04.349783 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:04.349750    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl" event={"ID":"a22688e9-f79b-469d-bcb9-8ea2edc86943","Type":"ContainerStarted","Data":"7f15011fb0433456edbdf35115276f83fde724fb69f42d2ce3f26d485d3589ad"}
Sep 15 11:45:04.351008 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:04.350991    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b8fbdd-29824545-bmjws"
Sep 15 11:45:04.351111 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:04.350991    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b8fbdd-29824545-bmjws" event={"ID":"b4e14abc-b9da-426e-9f2c-cdfc66c59637","Type":"ContainerDied","Data":"f7f8ed809aa0195eb7228a966aefba710115b18e895a5b9c7acceeeb948d90ae"}
Sep 15 11:45:04.351160 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:04.351119    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7f8ed809aa0195eb7228a966aefba710115b18e895a5b9c7acceeeb948d90ae"
Sep 15 11:45:04.352354 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:04.352334    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d63075-29824545-lwlw8"
Sep 15 11:45:04.352439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:04.352370    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d63075-29824545-lwlw8" event={"ID":"c106311e-7d48-411e-8eb2-7e539449d9a3","Type":"ContainerDied","Data":"739cf2a4f1f8adcde53cca5ffd34c3a15e13b3cc0a4c4794ea8f0e686769910d"}
Sep 15 11:45:04.352439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:04.352416    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="739cf2a4f1f8adcde53cca5ffd34c3a15e13b3cc0a4c4794ea8f0e686769910d"
Sep 15 11:45:04.353928 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:04.353910    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824545-jjjxz"
Sep 15 11:45:04.354036 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:04.353932    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29824545-jjjxz" event={"ID":"68f3ab4b-a83f-43e2-b96e-d90b3dab9a3d","Type":"ContainerDied","Data":"fadae8bf0317c51989bee589d4b5c462e77f140365240ebd75f9927821f1bfbf"}
Sep 15 11:45:04.354036 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:04.353973    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fadae8bf0317c51989bee589d4b5c462e77f140365240ebd75f9927821f1bfbf"
Sep 15 11:45:05.360931 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:05.360894    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl" event={"ID":"a22688e9-f79b-469d-bcb9-8ea2edc86943","Type":"ContainerStarted","Data":"7cb3305812505b7363b415bf48eb222498e020cb0c3f1113a684bf7b362d8f68"}
Sep 15 11:45:05.381514 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:05.381467    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl" podStartSLOduration=2.38145271 podStartE2EDuration="2.38145271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:45:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:45:05.379131842 +0000 UTC m=+1206.871908993" watchObservedRunningTime="2026-09-15 11:45:05.38145271 +0000 UTC m=+1206.874229858"
Sep 15 11:45:06.030173 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:06.030139    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:06.035889 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:06.035863    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:06.366760 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:06.366733    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:06.367780 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:06.367761    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:06.686778 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:06.686683    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:07.453241 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:07.453202    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:07.591090 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:07.591053    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:07.681183 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:07.681144    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:08.538518 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:08.538480    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:09.606881 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:09.606842    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:10.384664 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:10.384626    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:11.055498 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:11.055461    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:11.672163 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:11.672129    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"]
Sep 15 11:45:11.672356 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:11.672313    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv" podUID="6062a779-44ff-4a8e-b63a-5b41580f5913" containerName="istio-proxy" containerID="cri-o://6bfb112fab99e0c485a16f656148dec36c1ba59aa3172ddb9ac44090514496c6" gracePeriod=30
Sep 15 11:45:11.690408 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:11.690364    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:11.925934 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:11.925848    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:12.026708 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:12.026675    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:12.125152 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:12.125116    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:12.714920 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:12.714888    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:12.812998 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:12.812963    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:13.003290 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.003198    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:13.195792 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.195751    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:13.273925 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.273822    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:13.328436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.328404    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"]
Sep 15 11:45:13.333383 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.333365    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"
Sep 15 11:45:13.335524 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.335500    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-baee2b93\""
Sep 15 11:45:13.335629 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.335547    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-baee2b93-dockercfg-tbxb7\""
Sep 15 11:45:13.345047 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.345017    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"]
Sep 15 11:45:13.349511 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.349483    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f7fbec2-e489-48d2-97ce-dd2190f5368f-plugins-config-volume\") pod \"payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r\" (UID: \"4f7fbec2-e489-48d2-97ce-dd2190f5368f\") " pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"
Sep 15 11:45:13.349693 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.349674    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-257l2\" (UniqueName: \"kubernetes.io/projected/4f7fbec2-e489-48d2-97ce-dd2190f5368f-kube-api-access-257l2\") pod \"payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r\" (UID: \"4f7fbec2-e489-48d2-97ce-dd2190f5368f\") " pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"
Sep 15 11:45:13.454553 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.450481    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-257l2\" (UniqueName: \"kubernetes.io/projected/4f7fbec2-e489-48d2-97ce-dd2190f5368f-kube-api-access-257l2\") pod \"payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r\" (UID: \"4f7fbec2-e489-48d2-97ce-dd2190f5368f\") " pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"
Sep 15 11:45:13.454553 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.450618    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f7fbec2-e489-48d2-97ce-dd2190f5368f-plugins-config-volume\") pod \"payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r\" (UID: \"4f7fbec2-e489-48d2-97ce-dd2190f5368f\") " pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"
Sep 15 11:45:13.454553 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.451537    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f7fbec2-e489-48d2-97ce-dd2190f5368f-plugins-config-volume\") pod \"payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r\" (UID: \"4f7fbec2-e489-48d2-97ce-dd2190f5368f\") " pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"
Sep 15 11:45:13.472616 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.472405    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-257l2\" (UniqueName: \"kubernetes.io/projected/4f7fbec2-e489-48d2-97ce-dd2190f5368f-kube-api-access-257l2\") pod \"payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r\" (UID: \"4f7fbec2-e489-48d2-97ce-dd2190f5368f\") " pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"
Sep 15 11:45:13.645175 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.645132    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"
Sep 15 11:45:13.921969 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:13.921942    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"]
Sep 15 11:45:13.923433 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:45:13.923406    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f7fbec2_e489_48d2_97ce_dd2190f5368f.slice/crio-301f4158d89e16d80a415e7b8daef1f62f8ae840ef597c9343f8bd2ebf687046 WatchSource:0}: Error finding container 301f4158d89e16d80a415e7b8daef1f62f8ae840ef597c9343f8bd2ebf687046: Status 404 returned error can't find the container with id 301f4158d89e16d80a415e7b8daef1f62f8ae840ef597c9343f8bd2ebf687046
Sep 15 11:45:14.462747 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:14.462710    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r" event={"ID":"4f7fbec2-e489-48d2-97ce-dd2190f5368f","Type":"ContainerStarted","Data":"9547c6359099b571043aa7fa0001e4f4b2323f42f643e4c80e409c54d687da83"}
Sep 15 11:45:14.463179 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:14.462761    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r" event={"ID":"4f7fbec2-e489-48d2-97ce-dd2190f5368f","Type":"ContainerStarted","Data":"301f4158d89e16d80a415e7b8daef1f62f8ae840ef597c9343f8bd2ebf687046"}
Sep 15 11:45:14.463179 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:14.462780    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"
Sep 15 11:45:14.479480 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:14.479432    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r" podStartSLOduration=1.479417324 podStartE2EDuration="1.479417324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:45:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:45:14.478625998 +0000 UTC m=+1215.971403146" watchObservedRunningTime="2026-09-15 11:45:14.479417324 +0000 UTC m=+1215.972194473"
Sep 15 11:45:16.938939 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.938878    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:45:16.987331 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.987303    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-envoy\") pod \"6062a779-44ff-4a8e-b63a-5b41580f5913\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") "
Sep 15 11:45:16.987506 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.987361    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-data\") pod \"6062a779-44ff-4a8e-b63a-5b41580f5913\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") "
Sep 15 11:45:16.987506 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.987396    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfb5m\" (UniqueName: \"kubernetes.io/projected/6062a779-44ff-4a8e-b63a-5b41580f5913-kube-api-access-bfb5m\") pod \"6062a779-44ff-4a8e-b63a-5b41580f5913\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") "
Sep 15 11:45:16.987506 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.987434    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-credential-socket\") pod \"6062a779-44ff-4a8e-b63a-5b41580f5913\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") "
Sep 15 11:45:16.987506 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.987483    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-token\") pod \"6062a779-44ff-4a8e-b63a-5b41580f5913\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") "
Sep 15 11:45:16.987754 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.987548    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-workload-socket\") pod \"6062a779-44ff-4a8e-b63a-5b41580f5913\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") "
Sep 15 11:45:16.987754 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.987604    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-workload-certs\") pod \"6062a779-44ff-4a8e-b63a-5b41580f5913\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") "
Sep 15 11:45:16.987754 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.987683    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-podinfo\") pod \"6062a779-44ff-4a8e-b63a-5b41580f5913\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") "
Sep 15 11:45:16.987754 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.987752    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6062a779-44ff-4a8e-b63a-5b41580f5913-istiod-ca-cert\") pod \"6062a779-44ff-4a8e-b63a-5b41580f5913\" (UID: \"6062a779-44ff-4a8e-b63a-5b41580f5913\") "
Sep 15 11:45:16.989668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.988502    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6062a779-44ff-4a8e-b63a-5b41580f5913-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6062a779-44ff-4a8e-b63a-5b41580f5913" (UID: "6062a779-44ff-4a8e-b63a-5b41580f5913"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:45:16.989668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.988988    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6062a779-44ff-4a8e-b63a-5b41580f5913" (UID: "6062a779-44ff-4a8e-b63a-5b41580f5913"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:45:16.989668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.989255    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-data" (OuterVolumeSpecName: "istio-data") pod "6062a779-44ff-4a8e-b63a-5b41580f5913" (UID: "6062a779-44ff-4a8e-b63a-5b41580f5913"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:45:16.989668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.989551    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6062a779-44ff-4a8e-b63a-5b41580f5913" (UID: "6062a779-44ff-4a8e-b63a-5b41580f5913"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:45:16.989668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.989641    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6062a779-44ff-4a8e-b63a-5b41580f5913" (UID: "6062a779-44ff-4a8e-b63a-5b41580f5913"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:45:16.994580 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.994535    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-token" (OuterVolumeSpecName: "istio-token") pod "6062a779-44ff-4a8e-b63a-5b41580f5913" (UID: "6062a779-44ff-4a8e-b63a-5b41580f5913"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:45:16.994732 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.994707    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6062a779-44ff-4a8e-b63a-5b41580f5913-kube-api-access-bfb5m" (OuterVolumeSpecName: "kube-api-access-bfb5m") pod "6062a779-44ff-4a8e-b63a-5b41580f5913" (UID: "6062a779-44ff-4a8e-b63a-5b41580f5913"). InnerVolumeSpecName "kube-api-access-bfb5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:45:16.995466 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.995433    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6062a779-44ff-4a8e-b63a-5b41580f5913" (UID: "6062a779-44ff-4a8e-b63a-5b41580f5913"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:45:16.995584 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:16.995545    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6062a779-44ff-4a8e-b63a-5b41580f5913" (UID: "6062a779-44ff-4a8e-b63a-5b41580f5913"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:45:17.070940 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.070912    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:17.088787 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.088767    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:17.088787 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.088787    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:17.088787 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.088797    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:17.089021 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.088810    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:17.089021 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.088822    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6062a779-44ff-4a8e-b63a-5b41580f5913-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:17.089021 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.088834    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:17.089021 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.088843    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:17.089021 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.088852    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfb5m\" (UniqueName: \"kubernetes.io/projected/6062a779-44ff-4a8e-b63a-5b41580f5913-kube-api-access-bfb5m\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:17.089021 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.088862    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6062a779-44ff-4a8e-b63a-5b41580f5913-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:17.487609 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.487554    2560 generic.go:356] "Generic (PLEG): container finished" podID="6062a779-44ff-4a8e-b63a-5b41580f5913" containerID="6bfb112fab99e0c485a16f656148dec36c1ba59aa3172ddb9ac44090514496c6" exitCode=0
Sep 15 11:45:17.487872 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.487631    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"
Sep 15 11:45:17.488022 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.487647    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv" event={"ID":"6062a779-44ff-4a8e-b63a-5b41580f5913","Type":"ContainerDied","Data":"6bfb112fab99e0c485a16f656148dec36c1ba59aa3172ddb9ac44090514496c6"}
Sep 15 11:45:17.488192 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.488171    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv" event={"ID":"6062a779-44ff-4a8e-b63a-5b41580f5913","Type":"ContainerDied","Data":"be6635f000bad5c7f2ff2d1f9d537e6de0ed85c0c4db8ff71be97763e9cb7bde"}
Sep 15 11:45:17.488318 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.488304    2560 scope.go:117] "RemoveContainer" containerID="6bfb112fab99e0c485a16f656148dec36c1ba59aa3172ddb9ac44090514496c6"
Sep 15 11:45:17.502791 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.502763    2560 scope.go:117] "RemoveContainer" containerID="6bfb112fab99e0c485a16f656148dec36c1ba59aa3172ddb9ac44090514496c6"
Sep 15 11:45:17.503100 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:45:17.503080    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bfb112fab99e0c485a16f656148dec36c1ba59aa3172ddb9ac44090514496c6\": container with ID starting with 6bfb112fab99e0c485a16f656148dec36c1ba59aa3172ddb9ac44090514496c6 not found: ID does not exist" containerID="6bfb112fab99e0c485a16f656148dec36c1ba59aa3172ddb9ac44090514496c6"
Sep 15 11:45:17.503148 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.503107    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bfb112fab99e0c485a16f656148dec36c1ba59aa3172ddb9ac44090514496c6"} err="failed to get container status \"6bfb112fab99e0c485a16f656148dec36c1ba59aa3172ddb9ac44090514496c6\": rpc error: code = NotFound desc = could not find container \"6bfb112fab99e0c485a16f656148dec36c1ba59aa3172ddb9ac44090514496c6\": container with ID starting with 6bfb112fab99e0c485a16f656148dec36c1ba59aa3172ddb9ac44090514496c6 not found: ID does not exist"
Sep 15 11:45:17.508006 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.507985    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"]
Sep 15 11:45:17.513505 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:17.513482    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-ed9962-openshift-default-87699cbf4-km6hv"]
Sep 15 11:45:19.035541 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:19.035505    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6062a779-44ff-4a8e-b63a-5b41580f5913" path="/var/lib/kubelet/pods/6062a779-44ff-4a8e-b63a-5b41580f5913/volumes"
Sep 15 11:45:21.665657 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:21.665609    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:21.765464 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:21.765425    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:25.470342 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:25.470314    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"
Sep 15 11:45:32.583994 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:32.583917    2560 generic.go:356] "Generic (PLEG): container finished" podID="a22688e9-f79b-469d-bcb9-8ea2edc86943" containerID="7cb3305812505b7363b415bf48eb222498e020cb0c3f1113a684bf7b362d8f68" exitCode=0
Sep 15 11:45:32.583994 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:32.583953    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl" event={"ID":"a22688e9-f79b-469d-bcb9-8ea2edc86943","Type":"ContainerDied","Data":"7cb3305812505b7363b415bf48eb222498e020cb0c3f1113a684bf7b362d8f68"}
Sep 15 11:45:32.584430 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:32.584410    2560 scope.go:117] "RemoveContainer" containerID="7cb3305812505b7363b415bf48eb222498e020cb0c3f1113a684bf7b362d8f68"
Sep 15 11:45:32.585681 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:32.585651    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:45:32.585755 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:32.585732    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:45:32.585806 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:32.585765    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:45:33.594874 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:33.594826    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl" event={"ID":"a22688e9-f79b-469d-bcb9-8ea2edc86943","Type":"ContainerStarted","Data":"3471c778bf1e1411511a3fecb58a35f989544d2e081aa39f2073631f174dc27d"}
Sep 15 11:45:34.030491 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:34.030409    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:34.031553 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:34.031530    2560 patch_prober.go:28] interesting pod/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 15 11:45:34.031632 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:34.031602    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl" podUID="a22688e9-f79b-469d-bcb9-8ea2edc86943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 15 11:45:35.031096 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:35.031057    2560 patch_prober.go:28] interesting pod/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 15 11:45:35.031501 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:35.031129    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl" podUID="a22688e9-f79b-469d-bcb9-8ea2edc86943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 15 11:45:35.215208 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:35.215167    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:35.381621 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:35.381588    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk"]
Sep 15 11:45:35.382072 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:35.382058    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6062a779-44ff-4a8e-b63a-5b41580f5913" containerName="istio-proxy"
Sep 15 11:45:35.382138 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:35.382073    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6062a779-44ff-4a8e-b63a-5b41580f5913" containerName="istio-proxy"
Sep 15 11:45:35.382177 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:35.382162    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6062a779-44ff-4a8e-b63a-5b41580f5913" containerName="istio-proxy"
Sep 15 11:45:35.385499 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:35.385482    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk"
Sep 15 11:45:35.391295 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:35.391260    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbxjk\""
Sep 15 11:45:35.395145 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:35.395112    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk"]
Sep 15 11:45:35.493586 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:35.492138    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk\" (UID: \"7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk"
Sep 15 11:45:35.592936 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:35.592893    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk\" (UID: \"7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk"
Sep 15 11:45:35.593951 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:35.593920    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk\" (UID: \"7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk"
Sep 15 11:45:35.704485 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:35.704392    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk"
Sep 15 11:45:35.986899 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:35.986854    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk"]
Sep 15 11:45:36.030187 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:36.030154    2560 patch_prober.go:28] interesting pod/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 15 11:45:36.030285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:36.030207    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl" podUID="a22688e9-f79b-469d-bcb9-8ea2edc86943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 15 11:45:36.367652 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:36.367621    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:36.552026 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:36.551961    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:36.623777 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:36.623732    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk" event={"ID":"7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3","Type":"ContainerStarted","Data":"8fbd468f635382b12d39b3d6d4732e12b1e587e03ce54175a47c028440b32f98"}
Sep 15 11:45:36.623954 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:36.623802    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk" event={"ID":"7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3","Type":"ContainerStarted","Data":"b8ae901f84867fd1805ffd98071c2d3fab3c693f12f84dfa02779ef01f1e90bf"}
Sep 15 11:45:36.639889 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:36.639848    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk" podStartSLOduration=1.639831644 podStartE2EDuration="1.639831644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:45:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:45:36.638813918 +0000 UTC m=+1238.131591065" watchObservedRunningTime="2026-09-15 11:45:36.639831644 +0000 UTC m=+1238.132608793"
Sep 15 11:45:37.000356 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:37.000244    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:37.033282 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:37.033242    2560 patch_prober.go:28] interesting pod/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 15 11:45:37.033498 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:37.033314    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl" podUID="a22688e9-f79b-469d-bcb9-8ea2edc86943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 15 11:45:37.330706 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:37.330630    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:37.635958 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:37.635929    2560 generic.go:356] "Generic (PLEG): container finished" podID="7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3" containerID="8fbd468f635382b12d39b3d6d4732e12b1e587e03ce54175a47c028440b32f98" exitCode=0
Sep 15 11:45:37.636497 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:37.636076    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk" event={"ID":"7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3","Type":"ContainerDied","Data":"8fbd468f635382b12d39b3d6d4732e12b1e587e03ce54175a47c028440b32f98"}
Sep 15 11:45:37.907874 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:37.907782    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:38.031064 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:38.030937    2560 patch_prober.go:28] interesting pod/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 15 11:45:38.031064 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:38.030985    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl" podUID="a22688e9-f79b-469d-bcb9-8ea2edc86943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 15 11:45:38.657273 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:38.657192    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:38.807228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:38.807201    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk"
Sep 15 11:45:38.843906 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:38.843876    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3-maas-api-service-ca\") pod \"7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3\" (UID: \"7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3\") "
Sep 15 11:45:38.844311 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:38.844283    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3" (UID: "7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:45:38.945088 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:38.945021    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3-maas-api-service-ca\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:39.043968 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:39.043935    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:39.044843 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:39.044820    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:45:39.367194 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:39.367159    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:39.650004 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:39.649925    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk"
Sep 15 11:45:39.650004 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:39.649925    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk" event={"ID":"7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3","Type":"ContainerDied","Data":"b8ae901f84867fd1805ffd98071c2d3fab3c693f12f84dfa02779ef01f1e90bf"}
Sep 15 11:45:39.650191 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:39.650023    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8ae901f84867fd1805ffd98071c2d3fab3c693f12f84dfa02779ef01f1e90bf"
Sep 15 11:45:40.183933 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:40.183725    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:40.921058 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:40.921009    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:41.682482 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:41.682436    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:45.789622 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:45.788469    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"]
Sep 15 11:45:45.789622 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:45.788840    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz" podUID="cb53f9c3-1536-4739-aed4-49bc9814d1ec" containerName="maas-api" containerID="cri-o://c0161f46712e02d741ad535ee440a1bc0b23210ba0a737ea394e275303785fe7" gracePeriod=30
Sep 15 11:45:45.838310 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:45.838217    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:45.860321 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:45.860284    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"]
Sep 15 11:45:45.860579 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:45.860516    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb" podUID="3deab65e-55c8-4ca9-94a5-0e31678f10f9" containerName="payload-processing" containerID="cri-o://f346e56b178540107a2107832897f571c519cb043220925ec8f8ecc9bb35d554" gracePeriod=30
Sep 15 11:45:45.868250 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:45.868223    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"]
Sep 15 11:45:45.868445 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:45.868420    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6" podUID="7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b" containerName="payload-pre-processing" containerID="cri-o://7cdb0a99e29996a572d49480164bb2d450f051439468d3670a81aed2c82ee7bb" gracePeriod=30
Sep 15 11:45:45.910127 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:45.910089    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:46.010467 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.010234    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:46.031408 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.031345    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk"]
Sep 15 11:45:46.035515 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.035328    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9fc4b9-t7vvk"]
Sep 15 11:45:46.080276 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.080149    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:46.198021 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.197744    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:45:46.200223 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.200203    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"
Sep 15 11:45:46.210434 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.210407    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"
Sep 15 11:45:46.325450 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.325362    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3deab65e-55c8-4ca9-94a5-0e31678f10f9-plugins-config-volume\") pod \"3deab65e-55c8-4ca9-94a5-0e31678f10f9\" (UID: \"3deab65e-55c8-4ca9-94a5-0e31678f10f9\") "
Sep 15 11:45:46.325450 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.325410    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb53f9c3-1536-4739-aed4-49bc9814d1ec-openshift-service-ca\") pod \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") "
Sep 15 11:45:46.325450 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.325450    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fc54\" (UniqueName: \"kubernetes.io/projected/3deab65e-55c8-4ca9-94a5-0e31678f10f9-kube-api-access-2fc54\") pod \"3deab65e-55c8-4ca9-94a5-0e31678f10f9\" (UID: \"3deab65e-55c8-4ca9-94a5-0e31678f10f9\") "
Sep 15 11:45:46.325766 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.325609    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b-plugins-config-volume\") pod \"7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b\" (UID: \"7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b\") "
Sep 15 11:45:46.325766 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.325704    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fd2cf\" (UniqueName: \"kubernetes.io/projected/cb53f9c3-1536-4739-aed4-49bc9814d1ec-kube-api-access-fd2cf\") pod \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") "
Sep 15 11:45:46.325766 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.325728    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb53f9c3-1536-4739-aed4-49bc9814d1ec-metrics-certs\") pod \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") "
Sep 15 11:45:46.325766 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.325749    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69hgz\" (UniqueName: \"kubernetes.io/projected/7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b-kube-api-access-69hgz\") pod \"7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b\" (UID: \"7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b\") "
Sep 15 11:45:46.325999 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.325779    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb53f9c3-1536-4739-aed4-49bc9814d1ec-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cb53f9c3-1536-4739-aed4-49bc9814d1ec" (UID: "cb53f9c3-1536-4739-aed4-49bc9814d1ec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:45:46.325999 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.325799    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb53f9c3-1536-4739-aed4-49bc9814d1ec-maas-api-tls\") pod \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\" (UID: \"cb53f9c3-1536-4739-aed4-49bc9814d1ec\") "
Sep 15 11:45:46.326103 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.326013    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b" (UID: "7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:45:46.326231 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.326207    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3deab65e-55c8-4ca9-94a5-0e31678f10f9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3deab65e-55c8-4ca9-94a5-0e31678f10f9" (UID: "3deab65e-55c8-4ca9-94a5-0e31678f10f9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:45:46.326405 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.326351    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb53f9c3-1536-4739-aed4-49bc9814d1ec-openshift-service-ca\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:46.326405 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.326372    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:46.327939 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.327917    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b-kube-api-access-69hgz" (OuterVolumeSpecName: "kube-api-access-69hgz") pod "7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b" (UID: "7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b"). InnerVolumeSpecName "kube-api-access-69hgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:45:46.328177 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.328160    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3deab65e-55c8-4ca9-94a5-0e31678f10f9-kube-api-access-2fc54" (OuterVolumeSpecName: "kube-api-access-2fc54") pod "3deab65e-55c8-4ca9-94a5-0e31678f10f9" (UID: "3deab65e-55c8-4ca9-94a5-0e31678f10f9"). InnerVolumeSpecName "kube-api-access-2fc54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:45:46.328249 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.328195    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb53f9c3-1536-4739-aed4-49bc9814d1ec-kube-api-access-fd2cf" (OuterVolumeSpecName: "kube-api-access-fd2cf") pod "cb53f9c3-1536-4739-aed4-49bc9814d1ec" (UID: "cb53f9c3-1536-4739-aed4-49bc9814d1ec"). InnerVolumeSpecName "kube-api-access-fd2cf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:45:46.328340 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.328316    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb53f9c3-1536-4739-aed4-49bc9814d1ec-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cb53f9c3-1536-4739-aed4-49bc9814d1ec" (UID: "cb53f9c3-1536-4739-aed4-49bc9814d1ec"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:45:46.328404 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.328389    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb53f9c3-1536-4739-aed4-49bc9814d1ec-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cb53f9c3-1536-4739-aed4-49bc9814d1ec" (UID: "cb53f9c3-1536-4739-aed4-49bc9814d1ec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:45:46.372404 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.372363    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:46.427818 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.427788    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fd2cf\" (UniqueName: \"kubernetes.io/projected/cb53f9c3-1536-4739-aed4-49bc9814d1ec-kube-api-access-fd2cf\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:46.427818 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.427814    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb53f9c3-1536-4739-aed4-49bc9814d1ec-metrics-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:46.428006 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.427831    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69hgz\" (UniqueName: \"kubernetes.io/projected/7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b-kube-api-access-69hgz\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:46.428006 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.427842    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb53f9c3-1536-4739-aed4-49bc9814d1ec-maas-api-tls\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:46.428006 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.427853    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3deab65e-55c8-4ca9-94a5-0e31678f10f9-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:46.428006 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.427890    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fc54\" (UniqueName: \"kubernetes.io/projected/3deab65e-55c8-4ca9-94a5-0e31678f10f9-kube-api-access-2fc54\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:45:46.548646 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.548613    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:46.694459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.694423    2560 generic.go:356] "Generic (PLEG): container finished" podID="3deab65e-55c8-4ca9-94a5-0e31678f10f9" containerID="f346e56b178540107a2107832897f571c519cb043220925ec8f8ecc9bb35d554" exitCode=0
Sep 15 11:45:46.694696 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.694485    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"
Sep 15 11:45:46.694696 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.694517    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb" event={"ID":"3deab65e-55c8-4ca9-94a5-0e31678f10f9","Type":"ContainerDied","Data":"f346e56b178540107a2107832897f571c519cb043220925ec8f8ecc9bb35d554"}
Sep 15 11:45:46.694696 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.694593    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb" event={"ID":"3deab65e-55c8-4ca9-94a5-0e31678f10f9","Type":"ContainerDied","Data":"a780d99ce5cfada2e1fd983df406e6ada699af65fd76b4a4ac9d86818141cb11"}
Sep 15 11:45:46.694696 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.694623    2560 scope.go:117] "RemoveContainer" containerID="f346e56b178540107a2107832897f571c519cb043220925ec8f8ecc9bb35d554"
Sep 15 11:45:46.696056 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.696026    2560 generic.go:356] "Generic (PLEG): container finished" podID="cb53f9c3-1536-4739-aed4-49bc9814d1ec" containerID="c0161f46712e02d741ad535ee440a1bc0b23210ba0a737ea394e275303785fe7" exitCode=0
Sep 15 11:45:46.696178 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.696111    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"
Sep 15 11:45:46.696178 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.696132    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz" event={"ID":"cb53f9c3-1536-4739-aed4-49bc9814d1ec","Type":"ContainerDied","Data":"c0161f46712e02d741ad535ee440a1bc0b23210ba0a737ea394e275303785fe7"}
Sep 15 11:45:46.696302 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.696188    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz" event={"ID":"cb53f9c3-1536-4739-aed4-49bc9814d1ec","Type":"ContainerDied","Data":"1f58c9217da88283a19866cbb2a66f53bd642cdd287c4a6f08357215795c15c2"}
Sep 15 11:45:46.697851 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.697817    2560 generic.go:356] "Generic (PLEG): container finished" podID="7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b" containerID="7cdb0a99e29996a572d49480164bb2d450f051439468d3670a81aed2c82ee7bb" exitCode=0
Sep 15 11:45:46.697996 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.697857    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6" event={"ID":"7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b","Type":"ContainerDied","Data":"7cdb0a99e29996a572d49480164bb2d450f051439468d3670a81aed2c82ee7bb"}
Sep 15 11:45:46.697996 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.697873    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"
Sep 15 11:45:46.697996 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.697893    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6" event={"ID":"7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b","Type":"ContainerDied","Data":"3ae1eeaa021e7811c871f40d8e3174ac1bf9bc3b428ba352918471b55f0e3f75"}
Sep 15 11:45:46.706318 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.706261    2560 scope.go:117] "RemoveContainer" containerID="f346e56b178540107a2107832897f571c519cb043220925ec8f8ecc9bb35d554"
Sep 15 11:45:46.706622 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:45:46.706594    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f346e56b178540107a2107832897f571c519cb043220925ec8f8ecc9bb35d554\": container with ID starting with f346e56b178540107a2107832897f571c519cb043220925ec8f8ecc9bb35d554 not found: ID does not exist" containerID="f346e56b178540107a2107832897f571c519cb043220925ec8f8ecc9bb35d554"
Sep 15 11:45:46.706710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.706631    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f346e56b178540107a2107832897f571c519cb043220925ec8f8ecc9bb35d554"} err="failed to get container status \"f346e56b178540107a2107832897f571c519cb043220925ec8f8ecc9bb35d554\": rpc error: code = NotFound desc = could not find container \"f346e56b178540107a2107832897f571c519cb043220925ec8f8ecc9bb35d554\": container with ID starting with f346e56b178540107a2107832897f571c519cb043220925ec8f8ecc9bb35d554 not found: ID does not exist"
Sep 15 11:45:46.706710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.706653    2560 scope.go:117] "RemoveContainer" containerID="c0161f46712e02d741ad535ee440a1bc0b23210ba0a737ea394e275303785fe7"
Sep 15 11:45:46.715427 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.715412    2560 scope.go:117] "RemoveContainer" containerID="c0161f46712e02d741ad535ee440a1bc0b23210ba0a737ea394e275303785fe7"
Sep 15 11:45:46.715709 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:45:46.715690    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0161f46712e02d741ad535ee440a1bc0b23210ba0a737ea394e275303785fe7\": container with ID starting with c0161f46712e02d741ad535ee440a1bc0b23210ba0a737ea394e275303785fe7 not found: ID does not exist" containerID="c0161f46712e02d741ad535ee440a1bc0b23210ba0a737ea394e275303785fe7"
Sep 15 11:45:46.715781 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.715714    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0161f46712e02d741ad535ee440a1bc0b23210ba0a737ea394e275303785fe7"} err="failed to get container status \"c0161f46712e02d741ad535ee440a1bc0b23210ba0a737ea394e275303785fe7\": rpc error: code = NotFound desc = could not find container \"c0161f46712e02d741ad535ee440a1bc0b23210ba0a737ea394e275303785fe7\": container with ID starting with c0161f46712e02d741ad535ee440a1bc0b23210ba0a737ea394e275303785fe7 not found: ID does not exist"
Sep 15 11:45:46.715781 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.715729    2560 scope.go:117] "RemoveContainer" containerID="7cdb0a99e29996a572d49480164bb2d450f051439468d3670a81aed2c82ee7bb"
Sep 15 11:45:46.723378 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.723359    2560 scope.go:117] "RemoveContainer" containerID="7cdb0a99e29996a572d49480164bb2d450f051439468d3670a81aed2c82ee7bb"
Sep 15 11:45:46.723767 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:45:46.723710    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cdb0a99e29996a572d49480164bb2d450f051439468d3670a81aed2c82ee7bb\": container with ID starting with 7cdb0a99e29996a572d49480164bb2d450f051439468d3670a81aed2c82ee7bb not found: ID does not exist" containerID="7cdb0a99e29996a572d49480164bb2d450f051439468d3670a81aed2c82ee7bb"
Sep 15 11:45:46.723767 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.723746    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cdb0a99e29996a572d49480164bb2d450f051439468d3670a81aed2c82ee7bb"} err="failed to get container status \"7cdb0a99e29996a572d49480164bb2d450f051439468d3670a81aed2c82ee7bb\": rpc error: code = NotFound desc = could not find container \"7cdb0a99e29996a572d49480164bb2d450f051439468d3670a81aed2c82ee7bb\": container with ID starting with 7cdb0a99e29996a572d49480164bb2d450f051439468d3670a81aed2c82ee7bb not found: ID does not exist"
Sep 15 11:45:46.726524 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.726503    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"]
Sep 15 11:45:46.730936 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.730912    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9fc4b9-5b67fd497d-gzkt6"]
Sep 15 11:45:46.740888 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.740864    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"]
Sep 15 11:45:46.744847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.744827    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9fc4b9-57fd8f4b99-mgpnb"]
Sep 15 11:45:46.754649 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.754518    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"]
Sep 15 11:45:46.756942 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:46.756924    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9fc4b9-54b65d9767-2qwnz"]
Sep 15 11:45:47.034686 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:47.034612    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3deab65e-55c8-4ca9-94a5-0e31678f10f9" path="/var/lib/kubelet/pods/3deab65e-55c8-4ca9-94a5-0e31678f10f9/volumes"
Sep 15 11:45:47.035027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:47.034976    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b" path="/var/lib/kubelet/pods/7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b/volumes"
Sep 15 11:45:47.035267 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:47.035255    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3" path="/var/lib/kubelet/pods/7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3/volumes"
Sep 15 11:45:47.035642 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:47.035628    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb53f9c3-1536-4739-aed4-49bc9814d1ec" path="/var/lib/kubelet/pods/cb53f9c3-1536-4739-aed4-49bc9814d1ec/volumes"
Sep 15 11:45:47.223665 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:47.223626    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:48.075091 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:48.075059    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:48.869463 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:48.869429    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:49.250960 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:49.250868    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:49.495038 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:49.495001    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:50.327690 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:50.327657    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:50.973851 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:50.973804    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:51.405602 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:51.405544    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:53.700141 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:53.700101    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:54.629089 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:54.629038    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:55.745348 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.745313    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"]
Sep 15 11:45:55.745988 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.745965    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b" containerName="payload-pre-processing"
Sep 15 11:45:55.746071 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.745989    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b" containerName="payload-pre-processing"
Sep 15 11:45:55.746071 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.746021    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb53f9c3-1536-4739-aed4-49bc9814d1ec" containerName="maas-api"
Sep 15 11:45:55.746071 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.746030    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb53f9c3-1536-4739-aed4-49bc9814d1ec" containerName="maas-api"
Sep 15 11:45:55.746071 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.746059    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3deab65e-55c8-4ca9-94a5-0e31678f10f9" containerName="payload-processing"
Sep 15 11:45:55.746071 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.746068    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3deab65e-55c8-4ca9-94a5-0e31678f10f9" containerName="payload-processing"
Sep 15 11:45:55.746224 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.746080    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3" containerName="revoke-keys"
Sep 15 11:45:55.746224 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.746088    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3" containerName="revoke-keys"
Sep 15 11:45:55.746224 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.746175    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c39dbcc-2e09-4f4d-98dc-575b0ba78f7b" containerName="payload-pre-processing"
Sep 15 11:45:55.746224 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.746185    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb53f9c3-1536-4739-aed4-49bc9814d1ec" containerName="maas-api"
Sep 15 11:45:55.746224 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.746195    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e4bb34f-40ec-4e14-8f18-7ffa8f6485f3" containerName="revoke-keys"
Sep 15 11:45:55.746224 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.746205    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3deab65e-55c8-4ca9-94a5-0e31678f10f9" containerName="payload-processing"
Sep 15 11:45:55.751488 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.751467    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.753682 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.753658    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b8fbdd\"/\"openshift-service-ca.crt\""
Sep 15 11:45:55.753802 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.753658    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b8fbdd\"/\"kube-root-ca.crt\""
Sep 15 11:45:55.754136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.754110    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b8fbdd\"/\"default-dockercfg-62nm7\""
Sep 15 11:45:55.754248 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.754120    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b8fbdd\"/\"e2e-conflict-b-bc0d44e9-kserve-self-signed-certs\""
Sep 15 11:45:55.759312 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.759289    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"]
Sep 15 11:45:55.772810 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.772780    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:45:55.823063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.823030    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-tmp-dir\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.823231 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.823074    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-dshm\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.823231 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.823142    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-home\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.823231 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.823214    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-model-cache\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.823364 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.823237    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf159f3c-dd42-4c15-ada1-56570f84f018-tls-certs\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.823404 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.823367    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl8jp\" (UniqueName: \"kubernetes.io/projected/cf159f3c-dd42-4c15-ada1-56570f84f018-kube-api-access-kl8jp\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.925145 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.925086    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-tmp-dir\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.926220 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.925951    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-dshm\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.926460 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.926430    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-home\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.926750 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.925883    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-tmp-dir\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.926887 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.926862    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-home\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.927042 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.927002    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-model-cache\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.927123 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.927056    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf159f3c-dd42-4c15-ada1-56570f84f018-tls-certs\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.927246 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.927220    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl8jp\" (UniqueName: \"kubernetes.io/projected/cf159f3c-dd42-4c15-ada1-56570f84f018-kube-api-access-kl8jp\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.927322 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.927298    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-model-cache\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.930861 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.930837    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf159f3c-dd42-4c15-ada1-56570f84f018-tls-certs\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.936277 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.936240    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl8jp\" (UniqueName: \"kubernetes.io/projected/cf159f3c-dd42-4c15-ada1-56570f84f018-kube-api-access-kl8jp\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:55.936513 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:55.936488    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-dshm\") pod \"e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") " pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:56.068273 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:56.068183    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:45:56.344977 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:56.344951    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"]
Sep 15 11:45:56.346972 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:45:56.346943    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf159f3c_dd42_4c15_ada1_56570f84f018.slice/crio-a0d89516aa45c4b4134bacfa0a975e4f22297ff49a639f6017c73154941ef747 WatchSource:0}: Error finding container a0d89516aa45c4b4134bacfa0a975e4f22297ff49a639f6017c73154941ef747: Status 404 returned error can't find the container with id a0d89516aa45c4b4134bacfa0a975e4f22297ff49a639f6017c73154941ef747
Sep 15 11:45:56.759825 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:56.759728    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w" event={"ID":"cf159f3c-dd42-4c15-ada1-56570f84f018","Type":"ContainerStarted","Data":"94c6dc007a9283ba093a77d041687f40204709960c08e31692b41dd9d92c8665"}
Sep 15 11:45:56.759825 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:56.759773    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w" event={"ID":"cf159f3c-dd42-4c15-ada1-56570f84f018","Type":"ContainerStarted","Data":"a0d89516aa45c4b4134bacfa0a975e4f22297ff49a639f6017c73154941ef747"}
Sep 15 11:45:56.778362 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:45:56.778315    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w" podStartSLOduration=1.778300126 podStartE2EDuration="1.778300126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:45:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:45:56.776259651 +0000 UTC m=+1258.269036820" watchObservedRunningTime="2026-09-15 11:45:56.778300126 +0000 UTC m=+1258.271077311"
Sep 15 11:46:00.047302 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:00.047219    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:06.068709 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:06.068677    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:46:06.069095 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:06.068721    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:46:06.081048 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:06.081024    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:46:06.830699 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:06.830668    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:46:09.342450 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:09.342409    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:09.440555 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:09.440489    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:12.306792 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:12.306751    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:12.539984 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:12.539933    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:12.868687 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:12.868653    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k"]
Sep 15 11:46:12.873986 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:12.873965    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k"
Sep 15 11:46:12.876687 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:12.876665    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbxjk\""
Sep 15 11:46:12.879241 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:12.879213    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k"]
Sep 15 11:46:12.905414 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:12.905382    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b613502-f50a-4b5b-a345-6d62f3b8b01c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-78522e-lb26k\" (UID: \"4b613502-f50a-4b5b-a345-6d62f3b8b01c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k"
Sep 15 11:46:13.006549 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:13.006520    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b613502-f50a-4b5b-a345-6d62f3b8b01c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-78522e-lb26k\" (UID: \"4b613502-f50a-4b5b-a345-6d62f3b8b01c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k"
Sep 15 11:46:13.007147 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:13.007127    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b613502-f50a-4b5b-a345-6d62f3b8b01c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-78522e-lb26k\" (UID: \"4b613502-f50a-4b5b-a345-6d62f3b8b01c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k"
Sep 15 11:46:13.174910 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:13.174819    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:13.192977 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:13.192949    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k"
Sep 15 11:46:13.474345 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:13.474318    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k"]
Sep 15 11:46:13.861313 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:13.861280    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k" event={"ID":"4b613502-f50a-4b5b-a345-6d62f3b8b01c","Type":"ContainerStarted","Data":"177f13feb56cd9f8b23164dc931cb399f1ca27e5f98d1bc0443f6580f39a68da"}
Sep 15 11:46:13.861480 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:13.861329    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k" event={"ID":"4b613502-f50a-4b5b-a345-6d62f3b8b01c","Type":"ContainerStarted","Data":"f24a60eabfe7507e898b6183819d3682dc7b59028d383758d3e1015bd0e58bba"}
Sep 15 11:46:13.876728 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:13.876683    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k" podStartSLOduration=1.87666762 podStartE2EDuration="1.87666762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:46:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:46:13.875829512 +0000 UTC m=+1275.368606660" watchObservedRunningTime="2026-09-15 11:46:13.87666762 +0000 UTC m=+1275.369444768"
Sep 15 11:46:13.992443 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:13.992392    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:14.570472 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:14.570385    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:14.764744 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:14.764709    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:14.869436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:14.869398    2560 generic.go:356] "Generic (PLEG): container finished" podID="4b613502-f50a-4b5b-a345-6d62f3b8b01c" containerID="177f13feb56cd9f8b23164dc931cb399f1ca27e5f98d1bc0443f6580f39a68da" exitCode=0
Sep 15 11:46:14.869644 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:14.869489    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k" event={"ID":"4b613502-f50a-4b5b-a345-6d62f3b8b01c","Type":"ContainerDied","Data":"177f13feb56cd9f8b23164dc931cb399f1ca27e5f98d1bc0443f6580f39a68da"}
Sep 15 11:46:14.893613 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:14.893549    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:15.692617 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:15.692578    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:16.021233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:16.021210    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k"
Sep 15 11:46:16.139640 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:16.139610    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b613502-f50a-4b5b-a345-6d62f3b8b01c-maas-api-service-ca\") pod \"4b613502-f50a-4b5b-a345-6d62f3b8b01c\" (UID: \"4b613502-f50a-4b5b-a345-6d62f3b8b01c\") "
Sep 15 11:46:16.140027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:16.139999    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b613502-f50a-4b5b-a345-6d62f3b8b01c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4b613502-f50a-4b5b-a345-6d62f3b8b01c" (UID: "4b613502-f50a-4b5b-a345-6d62f3b8b01c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:46:16.241310 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:16.241272    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b613502-f50a-4b5b-a345-6d62f3b8b01c-maas-api-service-ca\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:46:16.469455 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:16.469379    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:16.882720 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:16.882687    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k"
Sep 15 11:46:16.883108 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:16.882693    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k" event={"ID":"4b613502-f50a-4b5b-a345-6d62f3b8b01c","Type":"ContainerDied","Data":"f24a60eabfe7507e898b6183819d3682dc7b59028d383758d3e1015bd0e58bba"}
Sep 15 11:46:16.883108 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:16.882811    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f24a60eabfe7507e898b6183819d3682dc7b59028d383758d3e1015bd0e58bba"
Sep 15 11:46:17.253424 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:17.253334    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:18.061629 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:18.061595    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:18.776226 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:18.776185    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:23.485616 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.485556    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:23.511881 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.511836    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"]
Sep 15 11:46:23.512119 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.512087    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf" podUID="e7b2de0a-ba15-4ea9-908a-4767e24e934b" containerName="payload-pre-processing" containerID="cri-o://7b504c078a99f39bedd46ad9ec95a32a169a2dae4124c1e5a95d8cb539b895a6" gracePeriod=30
Sep 15 11:46:23.582487 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.582451    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:23.665370 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.665328    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k"]
Sep 15 11:46:23.668334 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.668305    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-78522e-lb26k"]
Sep 15 11:46:23.669731 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.669699    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:23.731897 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.731861    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:23.808190 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.808108    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"
Sep 15 11:46:23.917283 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.917248    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7b2de0a-ba15-4ea9-908a-4767e24e934b-plugins-config-volume\") pod \"e7b2de0a-ba15-4ea9-908a-4767e24e934b\" (UID: \"e7b2de0a-ba15-4ea9-908a-4767e24e934b\") "
Sep 15 11:46:23.917490 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.917412    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkx6r\" (UniqueName: \"kubernetes.io/projected/e7b2de0a-ba15-4ea9-908a-4767e24e934b-kube-api-access-qkx6r\") pod \"e7b2de0a-ba15-4ea9-908a-4767e24e934b\" (UID: \"e7b2de0a-ba15-4ea9-908a-4767e24e934b\") "
Sep 15 11:46:23.917688 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.917662    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7b2de0a-ba15-4ea9-908a-4767e24e934b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e7b2de0a-ba15-4ea9-908a-4767e24e934b" (UID: "e7b2de0a-ba15-4ea9-908a-4767e24e934b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:46:23.917824 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.917808    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7b2de0a-ba15-4ea9-908a-4767e24e934b-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:46:23.919548 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.919522    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7b2de0a-ba15-4ea9-908a-4767e24e934b-kube-api-access-qkx6r" (OuterVolumeSpecName: "kube-api-access-qkx6r") pod "e7b2de0a-ba15-4ea9-908a-4767e24e934b" (UID: "e7b2de0a-ba15-4ea9-908a-4767e24e934b"). InnerVolumeSpecName "kube-api-access-qkx6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:46:23.924371 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.924338    2560 generic.go:356] "Generic (PLEG): container finished" podID="e7b2de0a-ba15-4ea9-908a-4767e24e934b" containerID="7b504c078a99f39bedd46ad9ec95a32a169a2dae4124c1e5a95d8cb539b895a6" exitCode=0
Sep 15 11:46:23.924472 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.924382    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"
Sep 15 11:46:23.924472 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.924420    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf" event={"ID":"e7b2de0a-ba15-4ea9-908a-4767e24e934b","Type":"ContainerDied","Data":"7b504c078a99f39bedd46ad9ec95a32a169a2dae4124c1e5a95d8cb539b895a6"}
Sep 15 11:46:23.924630 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.924484    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf" event={"ID":"e7b2de0a-ba15-4ea9-908a-4767e24e934b","Type":"ContainerDied","Data":"052228f3b68412545e4dc3be20cc20205c391e6f91a60408f598a6dda8c06257"}
Sep 15 11:46:23.924630 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.924511    2560 scope.go:117] "RemoveContainer" containerID="7b504c078a99f39bedd46ad9ec95a32a169a2dae4124c1e5a95d8cb539b895a6"
Sep 15 11:46:23.934720 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.934704    2560 scope.go:117] "RemoveContainer" containerID="7b504c078a99f39bedd46ad9ec95a32a169a2dae4124c1e5a95d8cb539b895a6"
Sep 15 11:46:23.934989 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:46:23.934972    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b504c078a99f39bedd46ad9ec95a32a169a2dae4124c1e5a95d8cb539b895a6\": container with ID starting with 7b504c078a99f39bedd46ad9ec95a32a169a2dae4124c1e5a95d8cb539b895a6 not found: ID does not exist" containerID="7b504c078a99f39bedd46ad9ec95a32a169a2dae4124c1e5a95d8cb539b895a6"
Sep 15 11:46:23.935080 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.934997    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b504c078a99f39bedd46ad9ec95a32a169a2dae4124c1e5a95d8cb539b895a6"} err="failed to get container status \"7b504c078a99f39bedd46ad9ec95a32a169a2dae4124c1e5a95d8cb539b895a6\": rpc error: code = NotFound desc = could not find container \"7b504c078a99f39bedd46ad9ec95a32a169a2dae4124c1e5a95d8cb539b895a6\": container with ID starting with 7b504c078a99f39bedd46ad9ec95a32a169a2dae4124c1e5a95d8cb539b895a6 not found: ID does not exist"
Sep 15 11:46:23.946725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.946700    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"]
Sep 15 11:46:23.952398 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:23.952380    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-78522e-666646fc98-n54hf"]
Sep 15 11:46:24.015820 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:24.015786    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:24.018912 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:24.018892    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkx6r\" (UniqueName: \"kubernetes.io/projected/e7b2de0a-ba15-4ea9-908a-4767e24e934b-kube-api-access-qkx6r\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:46:24.153485 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:24.153451    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:24.803033 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:24.802980    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:25.035003 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:25.034969    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b613502-f50a-4b5b-a345-6d62f3b8b01c" path="/var/lib/kubelet/pods/4b613502-f50a-4b5b-a345-6d62f3b8b01c/volumes"
Sep 15 11:46:25.035467 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:25.035446    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7b2de0a-ba15-4ea9-908a-4767e24e934b" path="/var/lib/kubelet/pods/e7b2de0a-ba15-4ea9-908a-4767e24e934b/volumes"
Sep 15 11:46:25.472165 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:25.472128    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:26.150961 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:26.150919    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:26.819515 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:26.819482    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:27.466446 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:27.466412    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:29.104118 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:29.104071    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:29.937003 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:29.936964    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:34.922572 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:34.922513    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:35.680536 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:35.680497    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:35.780949 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:35.780912    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:35.993475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:35.993387    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"]
Sep 15 11:46:35.994130 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:35.994105    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7b2de0a-ba15-4ea9-908a-4767e24e934b" containerName="payload-pre-processing"
Sep 15 11:46:35.994221 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:35.994132    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7b2de0a-ba15-4ea9-908a-4767e24e934b" containerName="payload-pre-processing"
Sep 15 11:46:35.994221 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:35.994177    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b613502-f50a-4b5b-a345-6d62f3b8b01c" containerName="revoke-keys"
Sep 15 11:46:35.994221 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:35.994188    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b613502-f50a-4b5b-a345-6d62f3b8b01c" containerName="revoke-keys"
Sep 15 11:46:35.994382 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:35.994294    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7b2de0a-ba15-4ea9-908a-4767e24e934b" containerName="payload-pre-processing"
Sep 15 11:46:35.994382 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:35.994311    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b613502-f50a-4b5b-a345-6d62f3b8b01c" containerName="revoke-keys"
Sep 15 11:46:35.997858 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:35.997827    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.001155 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.001133    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-7efa8b-openshift-default-dockercfg-75z67\""
Sep 15 11:46:36.008939 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.008915    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"]
Sep 15 11:46:36.026841 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.026808    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:36.139124 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.139088    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfjtq\" (UniqueName: \"kubernetes.io/projected/7e31b7b9-6f7b-403f-b310-551df113f4bb-kube-api-access-lfjtq\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.139317 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.139147    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-envoy\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.139317 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.139184    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-credential-socket\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.139317 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.139202    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-podinfo\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.139317 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.139219    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e31b7b9-6f7b-403f-b310-551df113f4bb-istiod-ca-cert\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.139317 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.139307    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-workload-socket\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.139642 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.139359    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-token\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.139642 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.139394    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-data\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.139642 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.139498    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-workload-certs\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.234908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.234856    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:36.240980 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.240956    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-envoy\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.241123 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.240991    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-credential-socket\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.241123 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.241010    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-podinfo\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.241123 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.241031    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e31b7b9-6f7b-403f-b310-551df113f4bb-istiod-ca-cert\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.241123 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.241049    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-workload-socket\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.241123 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.241080    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-token\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.241123 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.241100    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-data\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.241451 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.241170    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-workload-certs\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.241451 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.241269    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfjtq\" (UniqueName: \"kubernetes.io/projected/7e31b7b9-6f7b-403f-b310-551df113f4bb-kube-api-access-lfjtq\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.241451 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.241365    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-credential-socket\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.241630 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.241592    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-workload-certs\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.241694 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.241626    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-workload-socket\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.241821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.241796    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e31b7b9-6f7b-403f-b310-551df113f4bb-istiod-ca-cert\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.241821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.241817    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-data\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.243848 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.243802    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-envoy\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.244151 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.244132    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-podinfo\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.248819 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.248794    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfjtq\" (UniqueName: \"kubernetes.io/projected/7e31b7b9-6f7b-403f-b310-551df113f4bb-kube-api-access-lfjtq\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.248985 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.248967    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-token\") pod \"e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") " pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.310222 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.310186    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:36.585832 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.585803    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"]
Sep 15 11:46:36.587733 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:46:36.587702    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e31b7b9_6f7b_403f_b310_551df113f4bb.slice/crio-740bb576c2ea2f39fe0eb5108670e3af3bae50319ffa95b760437ea2c5959a2b WatchSource:0}: Error finding container 740bb576c2ea2f39fe0eb5108670e3af3bae50319ffa95b760437ea2c5959a2b: Status 404 returned error can't find the container with id 740bb576c2ea2f39fe0eb5108670e3af3bae50319ffa95b760437ea2c5959a2b
Sep 15 11:46:36.590756 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.590711    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:46:36.590873 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.590804    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:46:36.590873 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:36.590846    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:46:37.009866 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:37.009820    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" event={"ID":"7e31b7b9-6f7b-403f-b310-551df113f4bb","Type":"ContainerStarted","Data":"a352c3a93423fef478a7626e7ff220044df81328757af11bd1bb88a59ee8592e"}
Sep 15 11:46:37.010275 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:37.009890    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" event={"ID":"7e31b7b9-6f7b-403f-b310-551df113f4bb","Type":"ContainerStarted","Data":"740bb576c2ea2f39fe0eb5108670e3af3bae50319ffa95b760437ea2c5959a2b"}
Sep 15 11:46:37.030483 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:37.030437    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" podStartSLOduration=2.030421995 podStartE2EDuration="2.030421995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:46:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:46:37.028765665 +0000 UTC m=+1298.521542813" watchObservedRunningTime="2026-09-15 11:46:37.030421995 +0000 UTC m=+1298.523199143"
Sep 15 11:46:37.310681 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:37.310599    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:37.315540 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:37.315519    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:38.017774 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:38.017739    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:38.018784 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:38.018761    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:46:38.237509 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:38.237474    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:38.896373 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:38.896338    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:39.462783 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:39.462735    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:39.901737 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:39.901699    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:40.831906 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:40.831868    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:41.513669 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:41.513632    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:42.282674 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:42.282644    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:42.971612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:42.971572    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:43.034629 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:43.034598    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:43.583064 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:43.583030    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"]
Sep 15 11:46:43.590891 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:43.590865    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"
Sep 15 11:46:43.593138 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:43.593108    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-7efa8b\""
Sep 15 11:46:43.593138 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:43.593130    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-7efa8b-dockercfg-jssv6\""
Sep 15 11:46:43.599034 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:43.599009    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"]
Sep 15 11:46:43.671594 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:43.666592    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:43.721437 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:43.721396    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ca1834-cc47-44d1-9af2-91d5d92e4cc4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4\" (UID: \"12ca1834-cc47-44d1-9af2-91d5d92e4cc4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"
Sep 15 11:46:43.721650 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:43.721455    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4pnk\" (UniqueName: \"kubernetes.io/projected/12ca1834-cc47-44d1-9af2-91d5d92e4cc4-kube-api-access-j4pnk\") pod \"payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4\" (UID: \"12ca1834-cc47-44d1-9af2-91d5d92e4cc4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"
Sep 15 11:46:43.764738 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:43.764704    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:46:43.822173 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:43.822135    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4pnk\" (UniqueName: \"kubernetes.io/projected/12ca1834-cc47-44d1-9af2-91d5d92e4cc4-kube-api-access-j4pnk\") pod \"payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4\" (UID: \"12ca1834-cc47-44d1-9af2-91d5d92e4cc4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"
Sep 15 11:46:43.822340 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:43.822292    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ca1834-cc47-44d1-9af2-91d5d92e4cc4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4\" (UID: \"12ca1834-cc47-44d1-9af2-91d5d92e4cc4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"
Sep 15 11:46:43.823021 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:43.823000    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ca1834-cc47-44d1-9af2-91d5d92e4cc4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4\" (UID: \"12ca1834-cc47-44d1-9af2-91d5d92e4cc4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"
Sep 15 11:46:43.830108 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:43.830080    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4pnk\" (UniqueName: \"kubernetes.io/projected/12ca1834-cc47-44d1-9af2-91d5d92e4cc4-kube-api-access-j4pnk\") pod \"payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4\" (UID: \"12ca1834-cc47-44d1-9af2-91d5d92e4cc4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"
Sep 15 11:46:43.904966 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:43.904933    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"
Sep 15 11:46:44.179526 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:44.179500    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"]
Sep 15 11:46:44.181718 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:46:44.181692    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ca1834_cc47_44d1_9af2_91d5d92e4cc4.slice/crio-0242210f0b965dcab146816a970e85e9e0f734c5b559ac3ec7977f70d40ba741 WatchSource:0}: Error finding container 0242210f0b965dcab146816a970e85e9e0f734c5b559ac3ec7977f70d40ba741: Status 404 returned error can't find the container with id 0242210f0b965dcab146816a970e85e9e0f734c5b559ac3ec7977f70d40ba741
Sep 15 11:46:45.059208 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:45.059170    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4" event={"ID":"12ca1834-cc47-44d1-9af2-91d5d92e4cc4","Type":"ContainerStarted","Data":"58d19ab575bcf0d3e4636f503f58983b081824542d858cb14cbde0335ecbd1e6"}
Sep 15 11:46:45.059610 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:45.059237    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"
Sep 15 11:46:45.059610 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:45.059259    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4" event={"ID":"12ca1834-cc47-44d1-9af2-91d5d92e4cc4","Type":"ContainerStarted","Data":"0242210f0b965dcab146816a970e85e9e0f734c5b559ac3ec7977f70d40ba741"}
Sep 15 11:46:45.080174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:45.080131    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4" podStartSLOduration=2.080115702 podStartE2EDuration="2.080115702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:46:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:46:45.07847454 +0000 UTC m=+1306.571251689" watchObservedRunningTime="2026-09-15 11:46:45.080115702 +0000 UTC m=+1306.572892850"
Sep 15 11:46:56.067751 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:56.067721    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"
Sep 15 11:46:59.648397 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:46:59.648366    2560 scope.go:117] "RemoveContainer" containerID="3e871907d890f4bb369b09f750d63b9d6e52c850a6e4383a4380af2014ea1f77"
Sep 15 11:47:04.966584 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:04.966505    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:05.781131 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:05.781089    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:06.475520 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:06.475468    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:07.092514 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:07.092477    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:07.876442 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:07.876388    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:08.019668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:08.019549    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 15 11:47:08.019668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:08.019641    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 15 11:47:08.200350 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:08.200321    2560 generic.go:356] "Generic (PLEG): container finished" podID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerID="a352c3a93423fef478a7626e7ff220044df81328757af11bd1bb88a59ee8592e" exitCode=0
Sep 15 11:47:08.200518 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:08.200394    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" event={"ID":"7e31b7b9-6f7b-403f-b310-551df113f4bb","Type":"ContainerDied","Data":"a352c3a93423fef478a7626e7ff220044df81328757af11bd1bb88a59ee8592e"}
Sep 15 11:47:08.200853 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:08.200836    2560 scope.go:117] "RemoveContainer" containerID="a352c3a93423fef478a7626e7ff220044df81328757af11bd1bb88a59ee8592e"
Sep 15 11:47:08.202148 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:08.202117    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:47:08.202228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:08.202188    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:47:08.202228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:08.202222    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:47:08.579191 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:08.579087    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:08.955918 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:08.955887    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:09.209595 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.209489    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" event={"ID":"7e31b7b9-6f7b-403f-b310-551df113f4bb","Type":"ContainerStarted","Data":"5e9b930e1ea49a4205e8634466411d3f46d4a67e754d330ce0200341757639d0"}
Sep 15 11:47:09.219464 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.219441    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:09.311115 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.311082    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:47:09.311294 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.311267    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 15 11:47:09.311361 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.311328    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 15 11:47:09.337531 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.337497    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:09.580341 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.580256    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"]
Sep 15 11:47:09.584230 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.584204    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.586213 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.586187    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-2cc790-openshift-default-dockercfg-hwngg\""
Sep 15 11:47:09.595996 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.595968    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"]
Sep 15 11:47:09.604212 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.604186    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:09.698736 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.698699    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-token\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.698907 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.698774    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-data\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.698907 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.698817    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f9585cf-0d91-4424-946f-baf80cd5ca35-istiod-ca-cert\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.698907 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.698872    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-credential-socket\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.699037 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.698938    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-podinfo\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.699037 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.698978    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-envoy\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.699037 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.699011    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-workload-socket\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.699151 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.699037    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-workload-certs\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.699151 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.699083    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7csr\" (UniqueName: \"kubernetes.io/projected/4f9585cf-0d91-4424-946f-baf80cd5ca35-kube-api-access-q7csr\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.706807 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.706768    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:09.800181 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.800129    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-workload-socket\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.800398 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.800188    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-workload-certs\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.800398 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.800237    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7csr\" (UniqueName: \"kubernetes.io/projected/4f9585cf-0d91-4424-946f-baf80cd5ca35-kube-api-access-q7csr\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.800398 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.800310    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-token\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.800398 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.800365    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-data\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.800781 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.800406    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f9585cf-0d91-4424-946f-baf80cd5ca35-istiod-ca-cert\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.800781 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.800447    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-credential-socket\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.800781 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.800490    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-podinfo\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.800781 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.800527    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-envoy\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.804585 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.801675    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-workload-socket\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.804585 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.801922    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-workload-certs\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.804585 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.803029    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f9585cf-0d91-4424-946f-baf80cd5ca35-istiod-ca-cert\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.804585 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.803271    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-data\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.804585 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.803533    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-envoy\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.804585 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.803543    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-credential-socket\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.808629 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.806297    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-podinfo\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.823598 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.823518    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-token\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.823811 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.823707    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7csr\" (UniqueName: \"kubernetes.io/projected/4f9585cf-0d91-4424-946f-baf80cd5ca35-kube-api-access-q7csr\") pod \"e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") " pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.897296 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.897250    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:09.946853 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:09.946816    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:10.204647 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:10.204334    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"]
Sep 15 11:47:10.220094 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:47:10.220059    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f9585cf_0d91_4424_946f_baf80cd5ca35.slice/crio-618deed1dceae75a925718ef2182fe976b29c690b39abeec50d06bb8157a71dd WatchSource:0}: Error finding container 618deed1dceae75a925718ef2182fe976b29c690b39abeec50d06bb8157a71dd: Status 404 returned error can't find the container with id 618deed1dceae75a925718ef2182fe976b29c690b39abeec50d06bb8157a71dd
Sep 15 11:47:10.222383 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:10.222352    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:47:10.222455 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:10.222423    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:47:10.222493 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:10.222452    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:47:10.311622 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:10.311587    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 15 11:47:10.311793 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:10.311644    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 15 11:47:11.228401 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:11.228367    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj" event={"ID":"4f9585cf-0d91-4424-946f-baf80cd5ca35","Type":"ContainerStarted","Data":"1eafb4e205e7cc771ee92a6155c40fc546435862ac89d90dd927bacedaf35828"}
Sep 15 11:47:11.228866 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:11.228421    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj" event={"ID":"4f9585cf-0d91-4424-946f-baf80cd5ca35","Type":"ContainerStarted","Data":"618deed1dceae75a925718ef2182fe976b29c690b39abeec50d06bb8157a71dd"}
Sep 15 11:47:11.249784 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:11.249739    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj" podStartSLOduration=2.24972464 podStartE2EDuration="2.24972464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:47:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:47:11.247662065 +0000 UTC m=+1332.740439219" watchObservedRunningTime="2026-09-15 11:47:11.24972464 +0000 UTC m=+1332.742501787"
Sep 15 11:47:11.312241 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:11.312199    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 15 11:47:11.312433 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:11.312265    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 15 11:47:11.898067 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:11.898028    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:11.910140 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:11.910105    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:12.235811 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:12.235721    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:12.237524 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:12.237493    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:12.315587 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:12.311031    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 15 11:47:12.315587 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:12.311106    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 15 11:47:12.475738 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:12.475710    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:13.124058 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:13.124021    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:13.211752 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:13.211712    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:13.311586 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:13.311527    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 15 11:47:13.312764 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:13.312733    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 15 11:47:13.313695 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:13.313669    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:14.313358 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:14.312922    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 15 11:47:14.313358 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:14.312986    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 15 11:47:14.403522 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:14.403487    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:15.124487 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:15.124449    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:15.311325 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:15.311255    2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Sep 15 11:47:15.311525 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:15.311344    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Sep 15 11:47:15.800653 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:15.800617    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:16.314961 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:16.314931    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:47:16.315344 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:16.315316    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:47:16.316115 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:16.316098    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 11:47:16.628881 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:16.628845    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:17.359709 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:17.359677    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:18.094305 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:18.094268    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:18.161003 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:18.160967    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:18.409540 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:18.409442    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:18.672267 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:18.672182    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:18.766757 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:18.766723    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:39.404982 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:39.404946    2560 generic.go:356] "Generic (PLEG): container finished" podID="4f9585cf-0d91-4424-946f-baf80cd5ca35" containerID="1eafb4e205e7cc771ee92a6155c40fc546435862ac89d90dd927bacedaf35828" exitCode=0
Sep 15 11:47:39.405418 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:39.405026    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj" event={"ID":"4f9585cf-0d91-4424-946f-baf80cd5ca35","Type":"ContainerDied","Data":"1eafb4e205e7cc771ee92a6155c40fc546435862ac89d90dd927bacedaf35828"}
Sep 15 11:47:39.405493 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:39.405478    2560 scope.go:117] "RemoveContainer" containerID="1eafb4e205e7cc771ee92a6155c40fc546435862ac89d90dd927bacedaf35828"
Sep 15 11:47:39.406745 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:39.406712    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:47:39.406861 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:39.406797    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:47:39.406861 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:39.406850    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:47:40.402131 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:40.402088    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:40.417496 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:40.417445    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj" event={"ID":"4f9585cf-0d91-4424-946f-baf80cd5ca35","Type":"ContainerStarted","Data":"7e00fe2567e160ec018bebae171ff207b363804f2692279f9dc5f3ba9e9d974d"}
Sep 15 11:47:40.899450 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:40.899406    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:40.899694 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:40.899664    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 15 11:47:40.899765 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:40.899735    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj" podUID="4f9585cf-0d91-4424-946f-baf80cd5ca35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 15 11:47:41.309396 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:41.309296    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:41.898402 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:41.898355    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 15 11:47:41.898907 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:41.898429    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj" podUID="4f9585cf-0d91-4424-946f-baf80cd5ca35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 15 11:47:42.083038 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:42.082997    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:42.235978 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:42.235893    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:42.792355 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:42.792320    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:42.898344 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:42.898309    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 15 11:47:42.898673 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:42.898644    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj" podUID="4f9585cf-0d91-4424-946f-baf80cd5ca35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 15 11:47:43.281290 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:43.281261    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"]
Sep 15 11:47:43.286289 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:43.286259    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:43.354820 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:43.354795    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:43.592329 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:43.592292    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:43.898316 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:43.898220    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 15 11:47:43.898636 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:43.898591    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj" podUID="4f9585cf-0d91-4424-946f-baf80cd5ca35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 15 11:47:44.329240 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:44.329151    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:44.898831 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:44.898792    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 15 11:47:44.899139 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:44.899106    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj" podUID="4f9585cf-0d91-4424-946f-baf80cd5ca35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 15 11:47:45.172909 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:45.172820    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:45.818474 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:45.818430    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:47:45.902620 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:45.902591    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:45.904582 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:45.904534    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 11:47:54.505118 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:54.505086    2560 generic.go:356] "Generic (PLEG): container finished" podID="540f3743-3391-43a0-a98a-178d5e4cee78" containerID="519f7a86b0ec28b7e14b53c584cdfe61c4b8584ce32f7ce5b555063d6d30ec82" exitCode=137
Sep 15 11:47:54.505674 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:54.505165    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr" event={"ID":"540f3743-3391-43a0-a98a-178d5e4cee78","Type":"ContainerDied","Data":"519f7a86b0ec28b7e14b53c584cdfe61c4b8584ce32f7ce5b555063d6d30ec82"}
Sep 15 11:47:54.505674 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:54.505662    2560 scope.go:117] "RemoveContainer" containerID="519f7a86b0ec28b7e14b53c584cdfe61c4b8584ce32f7ce5b555063d6d30ec82"
Sep 15 11:47:55.513599 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:55.513545    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr" event={"ID":"540f3743-3391-43a0-a98a-178d5e4cee78","Type":"ContainerStarted","Data":"6207f467fa4e2836c40b53fd5f834ef02ac8ab8aecb9145c52910e99930e65eb"}
Sep 15 11:47:55.514040 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:55.513769    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"
Sep 15 11:47:58.334444 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.334382    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w" podUID="cf159f3c-dd42-4c15-ada1-56570f84f018" containerName="main" containerID="cri-o://94c6dc007a9283ba093a77d041687f40204709960c08e31692b41dd9d92c8665" gracePeriod=45
Sep 15 11:47:58.534033 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.534000    2560 generic.go:356] "Generic (PLEG): container finished" podID="cf159f3c-dd42-4c15-ada1-56570f84f018" containerID="94c6dc007a9283ba093a77d041687f40204709960c08e31692b41dd9d92c8665" exitCode=0
Sep 15 11:47:58.534247 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.534069    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w" event={"ID":"cf159f3c-dd42-4c15-ada1-56570f84f018","Type":"ContainerDied","Data":"94c6dc007a9283ba093a77d041687f40204709960c08e31692b41dd9d92c8665"}
Sep 15 11:47:58.596239 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.596214    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:47:58.723336 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.723303    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-dshm\") pod \"cf159f3c-dd42-4c15-ada1-56570f84f018\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") "
Sep 15 11:47:58.723510 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.723388    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf159f3c-dd42-4c15-ada1-56570f84f018-tls-certs\") pod \"cf159f3c-dd42-4c15-ada1-56570f84f018\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") "
Sep 15 11:47:58.723510 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.723420    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kl8jp\" (UniqueName: \"kubernetes.io/projected/cf159f3c-dd42-4c15-ada1-56570f84f018-kube-api-access-kl8jp\") pod \"cf159f3c-dd42-4c15-ada1-56570f84f018\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") "
Sep 15 11:47:58.723510 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.723440    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-tmp-dir\") pod \"cf159f3c-dd42-4c15-ada1-56570f84f018\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") "
Sep 15 11:47:58.723510 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.723458    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-home\") pod \"cf159f3c-dd42-4c15-ada1-56570f84f018\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") "
Sep 15 11:47:58.723732 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.723516    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-model-cache\") pod \"cf159f3c-dd42-4c15-ada1-56570f84f018\" (UID: \"cf159f3c-dd42-4c15-ada1-56570f84f018\") "
Sep 15 11:47:58.723792 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.723762    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cf159f3c-dd42-4c15-ada1-56570f84f018" (UID: "cf159f3c-dd42-4c15-ada1-56570f84f018"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:47:58.723869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.723812    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-home" (OuterVolumeSpecName: "home") pod "cf159f3c-dd42-4c15-ada1-56570f84f018" (UID: "cf159f3c-dd42-4c15-ada1-56570f84f018"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:47:58.723921 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.723874    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-model-cache" (OuterVolumeSpecName: "model-cache") pod "cf159f3c-dd42-4c15-ada1-56570f84f018" (UID: "cf159f3c-dd42-4c15-ada1-56570f84f018"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:47:58.724034 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.724019    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-tmp-dir\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:47:58.724080 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.724037    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-home\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:47:58.724080 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.724046    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-model-cache\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:47:58.725720 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.725694    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf159f3c-dd42-4c15-ada1-56570f84f018-kube-api-access-kl8jp" (OuterVolumeSpecName: "kube-api-access-kl8jp") pod "cf159f3c-dd42-4c15-ada1-56570f84f018" (UID: "cf159f3c-dd42-4c15-ada1-56570f84f018"). InnerVolumeSpecName "kube-api-access-kl8jp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:47:58.725982 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.725966    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf159f3c-dd42-4c15-ada1-56570f84f018-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cf159f3c-dd42-4c15-ada1-56570f84f018" (UID: "cf159f3c-dd42-4c15-ada1-56570f84f018"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:47:58.726041 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.725983    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-dshm" (OuterVolumeSpecName: "dshm") pod "cf159f3c-dd42-4c15-ada1-56570f84f018" (UID: "cf159f3c-dd42-4c15-ada1-56570f84f018"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:47:58.825419 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.825395    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl8jp\" (UniqueName: \"kubernetes.io/projected/cf159f3c-dd42-4c15-ada1-56570f84f018-kube-api-access-kl8jp\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:47:58.825419 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.825418    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf159f3c-dd42-4c15-ada1-56570f84f018-dshm\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:47:58.825554 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:58.825428    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf159f3c-dd42-4c15-ada1-56570f84f018-tls-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:47:59.540271 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:59.540242    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"
Sep 15 11:47:59.540787 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:59.540289    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w" event={"ID":"cf159f3c-dd42-4c15-ada1-56570f84f018","Type":"ContainerDied","Data":"a0d89516aa45c4b4134bacfa0a975e4f22297ff49a639f6017c73154941ef747"}
Sep 15 11:47:59.540787 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:59.540347    2560 scope.go:117] "RemoveContainer" containerID="94c6dc007a9283ba093a77d041687f40204709960c08e31692b41dd9d92c8665"
Sep 15 11:47:59.555058 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:59.555033    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"]
Sep 15 11:47:59.561508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:47:59.561485    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b8fbdd/e2e-conflict-b-bc0d44e9-kserve-7f9d7f7896-2zm5w"]
Sep 15 11:48:01.035329 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:01.035296    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf159f3c-dd42-4c15-ada1-56570f84f018" path="/var/lib/kubelet/pods/cf159f3c-dd42-4c15-ada1-56570f84f018/volumes"
Sep 15 11:48:03.803109 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:03.803055    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:06.523021 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:06.522990    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"
Sep 15 11:48:15.990890 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:15.990852    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:23.982641 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:23.982600    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:29.974547 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:29.974512    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq"]
Sep 15 11:48:29.975015 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:29.975000    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf159f3c-dd42-4c15-ada1-56570f84f018" containerName="main"
Sep 15 11:48:29.975015 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:29.975014    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf159f3c-dd42-4c15-ada1-56570f84f018" containerName="main"
Sep 15 11:48:29.975108 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:29.975088    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf159f3c-dd42-4c15-ada1-56570f84f018" containerName="main"
Sep 15 11:48:29.977787 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:29.977768    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq"
Sep 15 11:48:29.979628 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:29.979604    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbxjk\""
Sep 15 11:48:29.985500 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:29.985475    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq"]
Sep 15 11:48:30.136757 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:30.136724    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2262f4d9-5fee-41d7-8cd9-45b0a9468bb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq\" (UID: \"2262f4d9-5fee-41d7-8cd9-45b0a9468bb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq"
Sep 15 11:48:30.238178 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:30.238087    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2262f4d9-5fee-41d7-8cd9-45b0a9468bb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq\" (UID: \"2262f4d9-5fee-41d7-8cd9-45b0a9468bb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq"
Sep 15 11:48:30.238753 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:30.238732    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2262f4d9-5fee-41d7-8cd9-45b0a9468bb6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq\" (UID: \"2262f4d9-5fee-41d7-8cd9-45b0a9468bb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq"
Sep 15 11:48:30.290960 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:30.290920    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq"
Sep 15 11:48:30.370457 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:30.370425    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:30.570624 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:30.570599    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq"]
Sep 15 11:48:30.572753 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:48:30.572729    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2262f4d9_5fee_41d7_8cd9_45b0a9468bb6.slice/crio-6529d3a400c51850e8e0dc56e38db3f9e04245d8aae185df7908bcd3f7687ed9 WatchSource:0}: Error finding container 6529d3a400c51850e8e0dc56e38db3f9e04245d8aae185df7908bcd3f7687ed9: Status 404 returned error can't find the container with id 6529d3a400c51850e8e0dc56e38db3f9e04245d8aae185df7908bcd3f7687ed9
Sep 15 11:48:30.718431 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:30.718400    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq" event={"ID":"2262f4d9-5fee-41d7-8cd9-45b0a9468bb6","Type":"ContainerStarted","Data":"6529d3a400c51850e8e0dc56e38db3f9e04245d8aae185df7908bcd3f7687ed9"}
Sep 15 11:48:31.181674 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:31.181552    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:31.725965 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:31.725928    2560 generic.go:356] "Generic (PLEG): container finished" podID="2262f4d9-5fee-41d7-8cd9-45b0a9468bb6" containerID="1c826ba8d9f7b159c1aca96f31b587d4a11c2ba84f2cce4f46fee20d2ac5f66e" exitCode=0
Sep 15 11:48:31.726134 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:31.726010    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq" event={"ID":"2262f4d9-5fee-41d7-8cd9-45b0a9468bb6","Type":"ContainerDied","Data":"1c826ba8d9f7b159c1aca96f31b587d4a11c2ba84f2cce4f46fee20d2ac5f66e"}
Sep 15 11:48:31.817306 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:31.817215    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:32.484333 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:32.484279    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:32.886872 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:32.886840    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq"
Sep 15 11:48:33.070108 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:33.070026    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2262f4d9-5fee-41d7-8cd9-45b0a9468bb6-maas-api-service-ca\") pod \"2262f4d9-5fee-41d7-8cd9-45b0a9468bb6\" (UID: \"2262f4d9-5fee-41d7-8cd9-45b0a9468bb6\") "
Sep 15 11:48:33.070409 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:33.070383    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2262f4d9-5fee-41d7-8cd9-45b0a9468bb6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2262f4d9-5fee-41d7-8cd9-45b0a9468bb6" (UID: "2262f4d9-5fee-41d7-8cd9-45b0a9468bb6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:48:33.171096 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:33.171064    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2262f4d9-5fee-41d7-8cd9-45b0a9468bb6-maas-api-service-ca\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:48:33.181702 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:33.181665    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:33.740990 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:33.740962    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq"
Sep 15 11:48:33.741411 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:33.741007    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq" event={"ID":"2262f4d9-5fee-41d7-8cd9-45b0a9468bb6","Type":"ContainerDied","Data":"6529d3a400c51850e8e0dc56e38db3f9e04245d8aae185df7908bcd3f7687ed9"}
Sep 15 11:48:33.741411 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:33.741060    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6529d3a400c51850e8e0dc56e38db3f9e04245d8aae185df7908bcd3f7687ed9"
Sep 15 11:48:33.979498 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:33.979459    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:34.281088 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:34.281056    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:34.843395 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:34.843360    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:35.578550 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:35.578515    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:36.249958 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:36.249869    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:37.096267 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:37.096231    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:37.981080 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:37.981039    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:40.242551 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:40.242514    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:45.802390 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:45.802352    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:45.827308 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:45.827277    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"]
Sep 15 11:48:45.827532 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:45.827482    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r" podUID="4f7fbec2-e489-48d2-97ce-dd2190f5368f" containerName="payload-processing" containerID="cri-o://9547c6359099b571043aa7fa0001e4f4b2323f42f643e4c80e409c54d687da83" gracePeriod=30
Sep 15 11:48:45.901752 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:45.901697    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:46.001635 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.001594    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:46.033869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.033784    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq"]
Sep 15 11:48:46.037775 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.037739    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-baee2b93-9qrtq"]
Sep 15 11:48:46.072473 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.072394    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:46.139327 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.137691    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"
Sep 15 11:48:46.217598 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.217538    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-257l2\" (UniqueName: \"kubernetes.io/projected/4f7fbec2-e489-48d2-97ce-dd2190f5368f-kube-api-access-257l2\") pod \"4f7fbec2-e489-48d2-97ce-dd2190f5368f\" (UID: \"4f7fbec2-e489-48d2-97ce-dd2190f5368f\") "
Sep 15 11:48:46.217804 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.217692    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f7fbec2-e489-48d2-97ce-dd2190f5368f-plugins-config-volume\") pod \"4f7fbec2-e489-48d2-97ce-dd2190f5368f\" (UID: \"4f7fbec2-e489-48d2-97ce-dd2190f5368f\") "
Sep 15 11:48:46.218448 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.218084    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f7fbec2-e489-48d2-97ce-dd2190f5368f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4f7fbec2-e489-48d2-97ce-dd2190f5368f" (UID: "4f7fbec2-e489-48d2-97ce-dd2190f5368f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:48:46.218448 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.218169    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f7fbec2-e489-48d2-97ce-dd2190f5368f-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:48:46.220398 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.220370    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f7fbec2-e489-48d2-97ce-dd2190f5368f-kube-api-access-257l2" (OuterVolumeSpecName: "kube-api-access-257l2") pod "4f7fbec2-e489-48d2-97ce-dd2190f5368f" (UID: "4f7fbec2-e489-48d2-97ce-dd2190f5368f"). InnerVolumeSpecName "kube-api-access-257l2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:48:46.319400 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.319369    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-257l2\" (UniqueName: \"kubernetes.io/projected/4f7fbec2-e489-48d2-97ce-dd2190f5368f-kube-api-access-257l2\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:48:46.477007 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.476964    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:46.824754 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.824664    2560 generic.go:356] "Generic (PLEG): container finished" podID="4f7fbec2-e489-48d2-97ce-dd2190f5368f" containerID="9547c6359099b571043aa7fa0001e4f4b2323f42f643e4c80e409c54d687da83" exitCode=0
Sep 15 11:48:46.824754 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.824706    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r" event={"ID":"4f7fbec2-e489-48d2-97ce-dd2190f5368f","Type":"ContainerDied","Data":"9547c6359099b571043aa7fa0001e4f4b2323f42f643e4c80e409c54d687da83"}
Sep 15 11:48:46.824754 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.824713    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"
Sep 15 11:48:46.825308 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.824758    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r" event={"ID":"4f7fbec2-e489-48d2-97ce-dd2190f5368f","Type":"ContainerDied","Data":"301f4158d89e16d80a415e7b8daef1f62f8ae840ef597c9343f8bd2ebf687046"}
Sep 15 11:48:46.825308 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.824787    2560 scope.go:117] "RemoveContainer" containerID="9547c6359099b571043aa7fa0001e4f4b2323f42f643e4c80e409c54d687da83"
Sep 15 11:48:46.834812 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.834605    2560 scope.go:117] "RemoveContainer" containerID="9547c6359099b571043aa7fa0001e4f4b2323f42f643e4c80e409c54d687da83"
Sep 15 11:48:46.834945 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:48:46.834915    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9547c6359099b571043aa7fa0001e4f4b2323f42f643e4c80e409c54d687da83\": container with ID starting with 9547c6359099b571043aa7fa0001e4f4b2323f42f643e4c80e409c54d687da83 not found: ID does not exist" containerID="9547c6359099b571043aa7fa0001e4f4b2323f42f643e4c80e409c54d687da83"
Sep 15 11:48:46.834989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.834955    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9547c6359099b571043aa7fa0001e4f4b2323f42f643e4c80e409c54d687da83"} err="failed to get container status \"9547c6359099b571043aa7fa0001e4f4b2323f42f643e4c80e409c54d687da83\": rpc error: code = NotFound desc = could not find container \"9547c6359099b571043aa7fa0001e4f4b2323f42f643e4c80e409c54d687da83\": container with ID starting with 9547c6359099b571043aa7fa0001e4f4b2323f42f643e4c80e409c54d687da83 not found: ID does not exist"
Sep 15 11:48:46.846421 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.846397    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"]
Sep 15 11:48:46.850591 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:46.850554    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-baee2b93-54575b6977-rqn9r"]
Sep 15 11:48:47.038635 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:47.038602    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2262f4d9-5fee-41d7-8cd9-45b0a9468bb6" path="/var/lib/kubelet/pods/2262f4d9-5fee-41d7-8cd9-45b0a9468bb6/volumes"
Sep 15 11:48:47.039035 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:47.039020    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f7fbec2-e489-48d2-97ce-dd2190f5368f" path="/var/lib/kubelet/pods/4f7fbec2-e489-48d2-97ce-dd2190f5368f/volumes"
Sep 15 11:48:47.188946 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:47.188908    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:47.986842 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:47.986808    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:48.761071 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:48.761032    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:49.167856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:49.167814    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:49.398261 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:49.398217    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:50.132483 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:50.132439    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:50.793259 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:50.793201    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:54.237715 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:54.237637    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:48:59.690040 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:48:59.690004    2560 scope.go:117] "RemoveContainer" containerID="9b91e34a1473027f58cf7f670a52eab1ec44b8f87c3e42ba544b2b983989e5ec"
Sep 15 11:49:03.272487 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:03.272450    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:11.283822 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:11.283788    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:11.380172 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:11.380135    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:14.078002 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:14.077955    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:17.736119 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:17.736082    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:19.005544 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:19.005508    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:23.694845 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:23.694804    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:31.432578 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:31.432525    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:31.966197 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:31.966158    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:38.600664 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:38.600625    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:43.263109 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:43.263070    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:43.322422 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:43.322390    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:43.494100 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:43.494057    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"]
Sep 15 11:49:43.494868 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:43.494553    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl" podUID="a22688e9-f79b-469d-bcb9-8ea2edc86943" containerName="istio-proxy" containerID="cri-o://3471c778bf1e1411511a3fecb58a35f989544d2e081aa39f2073631f174dc27d" gracePeriod=30
Sep 15 11:49:43.507800 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:43.507765    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:44.773826 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:44.773788    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:44.873009 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:44.872959    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:44.973079 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:44.973042    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:45.230372 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:45.230336    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:45.542799 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:45.542705    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:45.640184 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:45.640151    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:45.740809 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:45.740768    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:46.968378 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:46.968342    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:48.761515 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.761454    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:49:48.763535 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.763513    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-envoy\") pod \"a22688e9-f79b-469d-bcb9-8ea2edc86943\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") "
Sep 15 11:49:48.763684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.763587    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-podinfo\") pod \"a22688e9-f79b-469d-bcb9-8ea2edc86943\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") "
Sep 15 11:49:48.763684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.763655    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a22688e9-f79b-469d-bcb9-8ea2edc86943-istiod-ca-cert\") pod \"a22688e9-f79b-469d-bcb9-8ea2edc86943\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") "
Sep 15 11:49:48.763684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.763674    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-credential-socket\") pod \"a22688e9-f79b-469d-bcb9-8ea2edc86943\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") "
Sep 15 11:49:48.763856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.763713    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-data\") pod \"a22688e9-f79b-469d-bcb9-8ea2edc86943\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") "
Sep 15 11:49:48.763856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.763733    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwt6s\" (UniqueName: \"kubernetes.io/projected/a22688e9-f79b-469d-bcb9-8ea2edc86943-kube-api-access-jwt6s\") pod \"a22688e9-f79b-469d-bcb9-8ea2edc86943\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") "
Sep 15 11:49:48.763856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.763771    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-workload-certs\") pod \"a22688e9-f79b-469d-bcb9-8ea2edc86943\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") "
Sep 15 11:49:48.763856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.763799    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-token\") pod \"a22688e9-f79b-469d-bcb9-8ea2edc86943\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") "
Sep 15 11:49:48.763856 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.763836    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-workload-socket\") pod \"a22688e9-f79b-469d-bcb9-8ea2edc86943\" (UID: \"a22688e9-f79b-469d-bcb9-8ea2edc86943\") "
Sep 15 11:49:48.764116 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.764001    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-data" (OuterVolumeSpecName: "istio-data") pod "a22688e9-f79b-469d-bcb9-8ea2edc86943" (UID: "a22688e9-f79b-469d-bcb9-8ea2edc86943"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:49:48.764116 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.764027    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a22688e9-f79b-469d-bcb9-8ea2edc86943-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a22688e9-f79b-469d-bcb9-8ea2edc86943" (UID: "a22688e9-f79b-469d-bcb9-8ea2edc86943"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:49:48.764116 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.764012    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a22688e9-f79b-469d-bcb9-8ea2edc86943" (UID: "a22688e9-f79b-469d-bcb9-8ea2edc86943"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:49:48.764275 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.764250    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a22688e9-f79b-469d-bcb9-8ea2edc86943" (UID: "a22688e9-f79b-469d-bcb9-8ea2edc86943"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:49:48.764425 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.764368    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a22688e9-f79b-469d-bcb9-8ea2edc86943-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:49:48.764425 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.764394    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:49:48.764425 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.764407    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:49:48.764425 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.764420    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:49:48.764815 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.764450    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a22688e9-f79b-469d-bcb9-8ea2edc86943" (UID: "a22688e9-f79b-469d-bcb9-8ea2edc86943"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:49:48.765888 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.765862    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a22688e9-f79b-469d-bcb9-8ea2edc86943" (UID: "a22688e9-f79b-469d-bcb9-8ea2edc86943"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:49:48.766284 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.766251    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a22688e9-f79b-469d-bcb9-8ea2edc86943" (UID: "a22688e9-f79b-469d-bcb9-8ea2edc86943"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:49:48.766284 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.766263    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a22688e9-f79b-469d-bcb9-8ea2edc86943-kube-api-access-jwt6s" (OuterVolumeSpecName: "kube-api-access-jwt6s") pod "a22688e9-f79b-469d-bcb9-8ea2edc86943" (UID: "a22688e9-f79b-469d-bcb9-8ea2edc86943"). InnerVolumeSpecName "kube-api-access-jwt6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:49:48.766284 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.766279    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-token" (OuterVolumeSpecName: "istio-token") pod "a22688e9-f79b-469d-bcb9-8ea2edc86943" (UID: "a22688e9-f79b-469d-bcb9-8ea2edc86943"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:49:48.865824 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.865790    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:49:48.865824 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.865823    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwt6s\" (UniqueName: \"kubernetes.io/projected/a22688e9-f79b-469d-bcb9-8ea2edc86943-kube-api-access-jwt6s\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:49:48.866042 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.865834    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:49:48.866042 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.865854    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:49:48.866042 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.865863    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a22688e9-f79b-469d-bcb9-8ea2edc86943-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:49:48.898808 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:48.898770    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:49.197551 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:49.197512    2560 generic.go:356] "Generic (PLEG): container finished" podID="a22688e9-f79b-469d-bcb9-8ea2edc86943" containerID="3471c778bf1e1411511a3fecb58a35f989544d2e081aa39f2073631f174dc27d" exitCode=0
Sep 15 11:49:49.197759 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:49.197590    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"
Sep 15 11:49:49.197759 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:49.197601    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl" event={"ID":"a22688e9-f79b-469d-bcb9-8ea2edc86943","Type":"ContainerDied","Data":"3471c778bf1e1411511a3fecb58a35f989544d2e081aa39f2073631f174dc27d"}
Sep 15 11:49:49.197759 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:49.197669    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl" event={"ID":"a22688e9-f79b-469d-bcb9-8ea2edc86943","Type":"ContainerDied","Data":"7f15011fb0433456edbdf35115276f83fde724fb69f42d2ce3f26d485d3589ad"}
Sep 15 11:49:49.197759 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:49.197702    2560 scope.go:117] "RemoveContainer" containerID="3471c778bf1e1411511a3fecb58a35f989544d2e081aa39f2073631f174dc27d"
Sep 15 11:49:49.207064 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:49.207039    2560 scope.go:117] "RemoveContainer" containerID="7cb3305812505b7363b415bf48eb222498e020cb0c3f1113a684bf7b362d8f68"
Sep 15 11:49:49.215437 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:49.215413    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"]
Sep 15 11:49:49.218107 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:49.218089    2560 scope.go:117] "RemoveContainer" containerID="3471c778bf1e1411511a3fecb58a35f989544d2e081aa39f2073631f174dc27d"
Sep 15 11:49:49.218433 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:49:49.218413    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3471c778bf1e1411511a3fecb58a35f989544d2e081aa39f2073631f174dc27d\": container with ID starting with 3471c778bf1e1411511a3fecb58a35f989544d2e081aa39f2073631f174dc27d not found: ID does not exist" containerID="3471c778bf1e1411511a3fecb58a35f989544d2e081aa39f2073631f174dc27d"
Sep 15 11:49:49.218511 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:49.218446    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3471c778bf1e1411511a3fecb58a35f989544d2e081aa39f2073631f174dc27d"} err="failed to get container status \"3471c778bf1e1411511a3fecb58a35f989544d2e081aa39f2073631f174dc27d\": rpc error: code = NotFound desc = could not find container \"3471c778bf1e1411511a3fecb58a35f989544d2e081aa39f2073631f174dc27d\": container with ID starting with 3471c778bf1e1411511a3fecb58a35f989544d2e081aa39f2073631f174dc27d not found: ID does not exist"
Sep 15 11:49:49.218511 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:49.218471    2560 scope.go:117] "RemoveContainer" containerID="7cb3305812505b7363b415bf48eb222498e020cb0c3f1113a684bf7b362d8f68"
Sep 15 11:49:49.218704 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:49.218683    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-baee2b93-openshift-default-7f96dd88b8-fkpgl"]
Sep 15 11:49:49.218897 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:49:49.218875    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cb3305812505b7363b415bf48eb222498e020cb0c3f1113a684bf7b362d8f68\": container with ID starting with 7cb3305812505b7363b415bf48eb222498e020cb0c3f1113a684bf7b362d8f68 not found: ID does not exist" containerID="7cb3305812505b7363b415bf48eb222498e020cb0c3f1113a684bf7b362d8f68"
Sep 15 11:49:49.218958 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:49.218905    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cb3305812505b7363b415bf48eb222498e020cb0c3f1113a684bf7b362d8f68"} err="failed to get container status \"7cb3305812505b7363b415bf48eb222498e020cb0c3f1113a684bf7b362d8f68\": rpc error: code = NotFound desc = could not find container \"7cb3305812505b7363b415bf48eb222498e020cb0c3f1113a684bf7b362d8f68\": container with ID starting with 7cb3305812505b7363b415bf48eb222498e020cb0c3f1113a684bf7b362d8f68 not found: ID does not exist"
Sep 15 11:49:51.035180 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:51.035147    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a22688e9-f79b-469d-bcb9-8ea2edc86943" path="/var/lib/kubelet/pods/a22688e9-f79b-469d-bcb9-8ea2edc86943/volumes"
Sep 15 11:49:53.045948 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:53.045912    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:53.114605 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:53.114556    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:53.620115 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:53.620079    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:59.644667 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:59.644631    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:49:59.647234 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:59.647210    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:49:59.716996 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:59.716967    2560 scope.go:117] "RemoveContainer" containerID="26c7af9e516452ea688b4429726b45513f0d9052047623c689c999b8955cfff8"
Sep 15 11:49:59.736276 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:59.736248    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:59.806430 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:59.806396    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:59.885101 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:59.885068    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:49:59.990306 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:49:59.990218    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:10.402363 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:10.402320    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:10.650915 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:10.650877    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:22.656125 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.656040    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"]
Sep 15 11:50:22.656911 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.656888    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2262f4d9-5fee-41d7-8cd9-45b0a9468bb6" containerName="revoke-keys"
Sep 15 11:50:22.657010 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.656913    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2262f4d9-5fee-41d7-8cd9-45b0a9468bb6" containerName="revoke-keys"
Sep 15 11:50:22.657010 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.656925    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a22688e9-f79b-469d-bcb9-8ea2edc86943" containerName="istio-proxy"
Sep 15 11:50:22.657010 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.656934    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a22688e9-f79b-469d-bcb9-8ea2edc86943" containerName="istio-proxy"
Sep 15 11:50:22.657010 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.656944    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f7fbec2-e489-48d2-97ce-dd2190f5368f" containerName="payload-processing"
Sep 15 11:50:22.657010 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.656952    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f7fbec2-e489-48d2-97ce-dd2190f5368f" containerName="payload-processing"
Sep 15 11:50:22.657010 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.656963    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a22688e9-f79b-469d-bcb9-8ea2edc86943" containerName="istio-proxy"
Sep 15 11:50:22.657010 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.656970    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a22688e9-f79b-469d-bcb9-8ea2edc86943" containerName="istio-proxy"
Sep 15 11:50:22.657351 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.657110    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a22688e9-f79b-469d-bcb9-8ea2edc86943" containerName="istio-proxy"
Sep 15 11:50:22.657351 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.657124    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2262f4d9-5fee-41d7-8cd9-45b0a9468bb6" containerName="revoke-keys"
Sep 15 11:50:22.657351 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.657140    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f7fbec2-e489-48d2-97ce-dd2190f5368f" containerName="payload-processing"
Sep 15 11:50:22.661753 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.661731    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.664500 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.664480    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-6e996c\"/\"default-dockercfg-hqdd9\""
Sep 15 11:50:22.664633 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.664482    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-6e996c\"/\"kube-root-ca.crt\""
Sep 15 11:50:22.664633 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.664528    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-6e996c\"/\"openshift-service-ca.crt\""
Sep 15 11:50:22.664633 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.664482    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-6e996c\"/\"embedding-w0-6e996c-kserve-self-signed-certs\""
Sep 15 11:50:22.671978 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.671958    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"]
Sep 15 11:50:22.684807 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.684765    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:22.709652 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.709624    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7186647-3a95-4239-a909-c02777b107c7-tls-certs\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.709783 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.709754    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-home\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.709833 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.709779    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsxc2\" (UniqueName: \"kubernetes.io/projected/e7186647-3a95-4239-a909-c02777b107c7-kube-api-access-lsxc2\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.709833 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.709822    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-model-cache\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.709920 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.709856    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-dshm\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.709920 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.709878    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-tmp-dir\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.810478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.810448    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-model-cache\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.810668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.810493    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-dshm\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.810668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.810521    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-tmp-dir\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.810668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.810554    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7186647-3a95-4239-a909-c02777b107c7-tls-certs\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.810798 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.810684    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-home\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.810798 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.810709    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsxc2\" (UniqueName: \"kubernetes.io/projected/e7186647-3a95-4239-a909-c02777b107c7-kube-api-access-lsxc2\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.810952 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.810926    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-model-cache\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.810997 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.810954    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-tmp-dir\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.811038 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.810995    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-home\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.812867 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.812837    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-dshm\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.813064 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.813049    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7186647-3a95-4239-a909-c02777b107c7-tls-certs\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.821006 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.820965    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsxc2\" (UniqueName: \"kubernetes.io/projected/e7186647-3a95-4239-a909-c02777b107c7-kube-api-access-lsxc2\") pod \"embedding-w0-6e996c-kserve-6f874cd68-jllhm\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") " pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:22.977123 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:22.977043    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:23.252112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:23.252086    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"]
Sep 15 11:50:23.253699 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:50:23.253671    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7186647_3a95_4239_a909_c02777b107c7.slice/crio-15f9563823e63f67aadc9ba7a50853147de1faa4012312ca8234ac483de07b85 WatchSource:0}: Error finding container 15f9563823e63f67aadc9ba7a50853147de1faa4012312ca8234ac483de07b85: Status 404 returned error can't find the container with id 15f9563823e63f67aadc9ba7a50853147de1faa4012312ca8234ac483de07b85
Sep 15 11:50:23.403855 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:23.403821    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm" event={"ID":"e7186647-3a95-4239-a909-c02777b107c7","Type":"ContainerStarted","Data":"97a1959c92103bea162cf99b6762e4015ef2198c64c6bcbcdff0a3fce300a8b5"}
Sep 15 11:50:23.404007 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:23.403862    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm" event={"ID":"e7186647-3a95-4239-a909-c02777b107c7","Type":"ContainerStarted","Data":"15f9563823e63f67aadc9ba7a50853147de1faa4012312ca8234ac483de07b85"}
Sep 15 11:50:23.423008 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:23.422960    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm" podStartSLOduration=1.4229447 podStartE2EDuration="1.4229447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:50:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:50:23.420743042 +0000 UTC m=+1524.913520191" watchObservedRunningTime="2026-09-15 11:50:23.4229447 +0000 UTC m=+1524.915721847"
Sep 15 11:50:31.554878 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:31.554839    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:32.978185 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:32.978151    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:32.978594 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:32.978221    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:32.990972 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:32.990950    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:33.472132 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:33.472105    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:50:37.738848 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:37.738808    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:37.841078 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:37.841036    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:37.997380 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:37.997292    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:38.098869 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.098833    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:38.199653 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.199619    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:38.332711 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.332638    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"]
Sep 15 11:50:38.333171 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.333156    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a22688e9-f79b-469d-bcb9-8ea2edc86943" containerName="istio-proxy"
Sep 15 11:50:38.336388 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.336365    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.338616 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.338554    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-bcb732-openshift-default-dockercfg-hw9p9\""
Sep 15 11:50:38.348379 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.348357    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"]
Sep 15 11:50:38.359686 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.359653    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:38.469806 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.469768    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-envoy\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.470115 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.470091    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istiod-ca-cert\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.470300 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.470284    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtj62\" (UniqueName: \"kubernetes.io/projected/49fbd7b3-8b22-415d-8f2f-243ea054e36f-kube-api-access-wtj62\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.470466 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.470452    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-workload-certs\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.470620 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.470602    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-credential-socket\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.470800 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.470774    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-podinfo\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.470946 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.470819    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-data\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.470946 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.470852    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-workload-socket\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.470946 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.470911    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-token\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.571549 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.571511    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtj62\" (UniqueName: \"kubernetes.io/projected/49fbd7b3-8b22-415d-8f2f-243ea054e36f-kube-api-access-wtj62\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.571784 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.571556    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-workload-certs\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.571784 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.571620    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-credential-socket\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.571784 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.571676    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-podinfo\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.571784 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.571709    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-data\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.571784 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.571739    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-workload-socket\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.572056 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.571784    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-token\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.572056 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.571873    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-envoy\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.572056 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.571931    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istiod-ca-cert\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.572056 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.572040    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-workload-certs\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.572267 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.572056    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-credential-socket\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.572267 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.572127    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-data\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.572267 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.572176    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-workload-socket\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.572642 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.572620    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istiod-ca-cert\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.574136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.574117    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-envoy\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.574340 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.574321    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-podinfo\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.579788 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.579760    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-token\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.579895 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.579880    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtj62\" (UniqueName: \"kubernetes.io/projected/49fbd7b3-8b22-415d-8f2f-243ea054e36f-kube-api-access-wtj62\") pod \"e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") " pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.648269 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.648238    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:38.928093 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.928064    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"]
Sep 15 11:50:38.930109 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:50:38.930075    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49fbd7b3_8b22_415d_8f2f_243ea054e36f.slice/crio-0fa7f75812d4b1d738760669ad94f9c83e3ff306119df3743eafca884dabd892 WatchSource:0}: Error finding container 0fa7f75812d4b1d738760669ad94f9c83e3ff306119df3743eafca884dabd892: Status 404 returned error can't find the container with id 0fa7f75812d4b1d738760669ad94f9c83e3ff306119df3743eafca884dabd892
Sep 15 11:50:38.932552 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.932519    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:50:38.932659 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.932607    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:50:38.932659 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.932640    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:50:38.956220 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:38.956168    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:39.497499 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:39.497459    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8" event={"ID":"49fbd7b3-8b22-415d-8f2f-243ea054e36f","Type":"ContainerStarted","Data":"68884d0e87f30dc64403ca34329de406f8b6f969c27366fe540a37bda8242e62"}
Sep 15 11:50:39.497717 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:39.497517    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8" event={"ID":"49fbd7b3-8b22-415d-8f2f-243ea054e36f","Type":"ContainerStarted","Data":"0fa7f75812d4b1d738760669ad94f9c83e3ff306119df3743eafca884dabd892"}
Sep 15 11:50:39.519004 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:39.518955    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8" podStartSLOduration=1.5189408210000002 podStartE2EDuration="1.518940821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:50:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:50:39.517000285 +0000 UTC m=+1541.009777438" watchObservedRunningTime="2026-09-15 11:50:39.518940821 +0000 UTC m=+1541.011717968"
Sep 15 11:50:39.648579 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:39.648525    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:39.654091 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:39.654065    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:40.504114 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:40.504069    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:40.505130 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:40.505106    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:50:40.686414 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:40.686383    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:41.504963 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:41.504925    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:41.591400 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:41.591355    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:42.317042 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:42.317003    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:43.321497 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:43.321454    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:43.900372 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:43.900335    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:44.700792 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:44.700752    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:45.376383 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:45.376347    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:46.309764 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:46.309726    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:46.402752 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:46.402721    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:46.485340 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:46.485306    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:46.667657 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:46.667617    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:46.920315 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:46.920229    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:47.020484 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.020442    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:50:47.108404 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.108369    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"]
Sep 15 11:50:47.112513 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.112493    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"
Sep 15 11:50:47.120488 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.120458    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-bcb732\""
Sep 15 11:50:47.120768 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.120744    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-bcb732-dockercfg-b6dsn\""
Sep 15 11:50:47.125418 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.125396    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"]
Sep 15 11:50:47.203362 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.203274    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"]
Sep 15 11:50:47.216801 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.216769    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"]
Sep 15 11:50:47.217098 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.217084    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"
Sep 15 11:50:47.265443 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.265387    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/772effbf-c80e-4501-97ac-5fa9cdb067a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh\" (UID: \"772effbf-c80e-4501-97ac-5fa9cdb067a5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"
Sep 15 11:50:47.265668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.265490    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hnxb\" (UniqueName: \"kubernetes.io/projected/772effbf-c80e-4501-97ac-5fa9cdb067a5-kube-api-access-4hnxb\") pod \"payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh\" (UID: \"772effbf-c80e-4501-97ac-5fa9cdb067a5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"
Sep 15 11:50:47.366495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.366458    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hnxb\" (UniqueName: \"kubernetes.io/projected/772effbf-c80e-4501-97ac-5fa9cdb067a5-kube-api-access-4hnxb\") pod \"payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh\" (UID: \"772effbf-c80e-4501-97ac-5fa9cdb067a5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"
Sep 15 11:50:47.366979 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.366595    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ce9202b-2e85-448a-9811-d211f5da58a2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd\" (UID: \"5ce9202b-2e85-448a-9811-d211f5da58a2\") " pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"
Sep 15 11:50:47.366979 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.366746    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/772effbf-c80e-4501-97ac-5fa9cdb067a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh\" (UID: \"772effbf-c80e-4501-97ac-5fa9cdb067a5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"
Sep 15 11:50:47.366979 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.366805    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q89ln\" (UniqueName: \"kubernetes.io/projected/5ce9202b-2e85-448a-9811-d211f5da58a2-kube-api-access-q89ln\") pod \"payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd\" (UID: \"5ce9202b-2e85-448a-9811-d211f5da58a2\") " pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"
Sep 15 11:50:47.367471 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.367449    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/772effbf-c80e-4501-97ac-5fa9cdb067a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh\" (UID: \"772effbf-c80e-4501-97ac-5fa9cdb067a5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"
Sep 15 11:50:47.374828 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.374801    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hnxb\" (UniqueName: \"kubernetes.io/projected/772effbf-c80e-4501-97ac-5fa9cdb067a5-kube-api-access-4hnxb\") pod \"payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh\" (UID: \"772effbf-c80e-4501-97ac-5fa9cdb067a5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"
Sep 15 11:50:47.431520 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.431487    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"
Sep 15 11:50:47.468203 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.468126    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ce9202b-2e85-448a-9811-d211f5da58a2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd\" (UID: \"5ce9202b-2e85-448a-9811-d211f5da58a2\") " pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"
Sep 15 11:50:47.468355 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.468238    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q89ln\" (UniqueName: \"kubernetes.io/projected/5ce9202b-2e85-448a-9811-d211f5da58a2-kube-api-access-q89ln\") pod \"payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd\" (UID: \"5ce9202b-2e85-448a-9811-d211f5da58a2\") " pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"
Sep 15 11:50:47.468871 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.468847    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ce9202b-2e85-448a-9811-d211f5da58a2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd\" (UID: \"5ce9202b-2e85-448a-9811-d211f5da58a2\") " pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"
Sep 15 11:50:47.476719 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.476689    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q89ln\" (UniqueName: \"kubernetes.io/projected/5ce9202b-2e85-448a-9811-d211f5da58a2-kube-api-access-q89ln\") pod \"payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd\" (UID: \"5ce9202b-2e85-448a-9811-d211f5da58a2\") " pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"
Sep 15 11:50:47.542523 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.542230    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"
Sep 15 11:50:47.721197 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.721048    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"]
Sep 15 11:50:47.890612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:47.890579    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"]
Sep 15 11:50:47.892517 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:50:47.892279    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ce9202b_2e85_448a_9811_d211f5da58a2.slice/crio-d4382d970312a1a9867e66680e4656ced14950b408d9ad9712ac76f783fc54ab WatchSource:0}: Error finding container d4382d970312a1a9867e66680e4656ced14950b408d9ad9712ac76f783fc54ab: Status 404 returned error can't find the container with id d4382d970312a1a9867e66680e4656ced14950b408d9ad9712ac76f783fc54ab
Sep 15 11:50:48.561159 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:48.561114    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh" event={"ID":"772effbf-c80e-4501-97ac-5fa9cdb067a5","Type":"ContainerStarted","Data":"8cc3a61325a37d21b2919132ece3a985f717992ac277408618e91c672505878f"}
Sep 15 11:50:48.561641 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:48.561264    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"
Sep 15 11:50:48.561641 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:48.561315    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh" event={"ID":"772effbf-c80e-4501-97ac-5fa9cdb067a5","Type":"ContainerStarted","Data":"686ec542384d7df549e4025e3edd069b16f6fc99c853d48415c8a8b126255e40"}
Sep 15 11:50:48.562978 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:48.562952    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd" event={"ID":"5ce9202b-2e85-448a-9811-d211f5da58a2","Type":"ContainerStarted","Data":"01101d4322bd8783238d38384ebc5f0dce7766a720f2d9c5c4ac7e4db0c706e1"}
Sep 15 11:50:48.563103 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:48.562997    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd" event={"ID":"5ce9202b-2e85-448a-9811-d211f5da58a2","Type":"ContainerStarted","Data":"d4382d970312a1a9867e66680e4656ced14950b408d9ad9712ac76f783fc54ab"}
Sep 15 11:50:48.563103 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:48.563084    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"
Sep 15 11:50:48.578099 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:48.578049    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh" podStartSLOduration=1.578034451 podStartE2EDuration="1.578034451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:50:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:50:48.576397597 +0000 UTC m=+1550.069174746" watchObservedRunningTime="2026-09-15 11:50:48.578034451 +0000 UTC m=+1550.070811599"
Sep 15 11:50:48.594068 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:48.594023    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd" podStartSLOduration=1.594010907 podStartE2EDuration="1.594010907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:50:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:50:48.591946698 +0000 UTC m=+1550.084723850" watchObservedRunningTime="2026-09-15 11:50:48.594010907 +0000 UTC m=+1550.086788055"
Sep 15 11:50:59.571594 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:59.571549    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"
Sep 15 11:50:59.572083 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:50:59.571721    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"
Sep 15 11:51:09.785193 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:09.785153    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:10.548972 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:10.548932    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:11.406344 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:11.406300    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:11.527841 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:11.527805    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:11.953826 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:11.953789    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:12.231256 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.231169    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"]
Sep 15 11:51:12.235347 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.235322    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.237808 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.237779    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-2f0e97-openshift-default-dockercfg-66m27\""
Sep 15 11:51:12.248923 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.248896    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"]
Sep 15 11:51:12.268133 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.268100    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:12.346163 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.346130    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfpwh\" (UniqueName: \"kubernetes.io/projected/152bfca4-485b-4870-92fb-c41e53ed1e3d-kube-api-access-vfpwh\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.346334 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.346170    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-workload-certs\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.346334 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.346194    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-workload-socket\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.346334 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.346209    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-envoy\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.346334 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.346230    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-credential-socket\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.346334 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.346305    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-podinfo\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.346334 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.346320    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-token\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.346605 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.346343    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-data\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.346605 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.346437    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/152bfca4-485b-4870-92fb-c41e53ed1e3d-istiod-ca-cert\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.365988 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.365954    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:12.447748 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.447709    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-data\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.448292 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.448260    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-data\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.448449 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.448350    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/152bfca4-485b-4870-92fb-c41e53ed1e3d-istiod-ca-cert\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.448665 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.448644    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfpwh\" (UniqueName: \"kubernetes.io/projected/152bfca4-485b-4870-92fb-c41e53ed1e3d-kube-api-access-vfpwh\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.448799 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.448780    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-workload-certs\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.448916 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.448900    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-workload-socket\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.449012 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.449000    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-envoy\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.449112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.449097    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-credential-socket\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.449258 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.449244    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-podinfo\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.449357 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.449344    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-token\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.449621 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.449051    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/152bfca4-485b-4870-92fb-c41e53ed1e3d-istiod-ca-cert\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.449918 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.449894    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-workload-socket\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.450007 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.449286    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-workload-certs\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.450390 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.450368    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-credential-socket\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.455617 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.454130    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-podinfo\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.475924 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.475883    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-envoy\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.479097 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.479063    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-token\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.480012 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.479941    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfpwh\" (UniqueName: \"kubernetes.io/projected/152bfca4-485b-4870-92fb-c41e53ed1e3d-kube-api-access-vfpwh\") pod \"e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") " pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.552423 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.552345    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:12.684513 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.684483    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:12.866362 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.866333    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"]
Sep 15 11:51:12.867217 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:51:12.867188    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod152bfca4_485b_4870_92fb_c41e53ed1e3d.slice/crio-bb8ce7f4ed55a27f1f8007e97fe92c3308726d9b486afc46b409617dd38b8485 WatchSource:0}: Error finding container bb8ce7f4ed55a27f1f8007e97fe92c3308726d9b486afc46b409617dd38b8485: Status 404 returned error can't find the container with id bb8ce7f4ed55a27f1f8007e97fe92c3308726d9b486afc46b409617dd38b8485
Sep 15 11:51:12.869807 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.869775    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:51:12.869924 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.869863    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:51:12.869924 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:12.869909    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:51:13.226179 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:13.226085    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:13.732450 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:13.732407    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw" event={"ID":"152bfca4-485b-4870-92fb-c41e53ed1e3d","Type":"ContainerStarted","Data":"4185c467f67c2335e39894010b64c78275ab539d9d2a8baca563c7868e391526"}
Sep 15 11:51:13.732925 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:13.732479    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw" event={"ID":"152bfca4-485b-4870-92fb-c41e53ed1e3d","Type":"ContainerStarted","Data":"bb8ce7f4ed55a27f1f8007e97fe92c3308726d9b486afc46b409617dd38b8485"}
Sep 15 11:51:13.753703 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:13.753653    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw" podStartSLOduration=1.753636584 podStartE2EDuration="1.753636584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:51:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:51:13.75175671 +0000 UTC m=+1575.244533851" watchObservedRunningTime="2026-09-15 11:51:13.753636584 +0000 UTC m=+1575.246413731"
Sep 15 11:51:14.111472 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:14.111436    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:14.553425 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:14.553336    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:14.559489 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:14.559458    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:14.740601 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:14.740573    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:14.741543 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:14.741520    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:15.112381 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:15.112342    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:15.782848 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:15.782819    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:15.940847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:15.940801    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:16.200914 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:16.200882    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:17.003675 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.003628    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:17.105975 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.105937    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:17.115040 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.115011    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4"]
Sep 15 11:51:17.119793 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.119772    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4"
Sep 15 11:51:17.122397 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.122375    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbxjk\""
Sep 15 11:51:17.127073 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.127050    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4"]
Sep 15 11:51:17.188671 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.177731    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:17.207233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.207179    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/908fc30e-29b6-4090-9e55-66612d32bf59-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4\" (UID: \"908fc30e-29b6-4090-9e55-66612d32bf59\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4"
Sep 15 11:51:17.308420 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.308335    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/908fc30e-29b6-4090-9e55-66612d32bf59-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4\" (UID: \"908fc30e-29b6-4090-9e55-66612d32bf59\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4"
Sep 15 11:51:17.309045 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.309018    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/908fc30e-29b6-4090-9e55-66612d32bf59-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4\" (UID: \"908fc30e-29b6-4090-9e55-66612d32bf59\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4"
Sep 15 11:51:17.436341 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.436304    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4"
Sep 15 11:51:17.523927 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.523877    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:17.738434 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.738269    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4"]
Sep 15 11:51:17.742914 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:51:17.742876    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod908fc30e_29b6_4090_9e55_66612d32bf59.slice/crio-9c1d0ca617e835ee81311353d4d3fe19c8209d56cf265c2ab83e65be083383b5 WatchSource:0}: Error finding container 9c1d0ca617e835ee81311353d4d3fe19c8209d56cf265c2ab83e65be083383b5: Status 404 returned error can't find the container with id 9c1d0ca617e835ee81311353d4d3fe19c8209d56cf265c2ab83e65be083383b5
Sep 15 11:51:17.759153 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.759075    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:17.763690 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.763658    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4" event={"ID":"908fc30e-29b6-4090-9e55-66612d32bf59","Type":"ContainerStarted","Data":"9c1d0ca617e835ee81311353d4d3fe19c8209d56cf265c2ab83e65be083383b5"}
Sep 15 11:51:17.831925 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:17.831876    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:18.341742 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:18.341708    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:18.772037 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:18.772004    2560 generic.go:356] "Generic (PLEG): container finished" podID="908fc30e-29b6-4090-9e55-66612d32bf59" containerID="41bab9d928c6ae739b628d83306966f83f3a5b714ebf2ea5770a4d57640b438f" exitCode=0
Sep 15 11:51:18.772167 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:18.772057    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4" event={"ID":"908fc30e-29b6-4090-9e55-66612d32bf59","Type":"ContainerDied","Data":"41bab9d928c6ae739b628d83306966f83f3a5b714ebf2ea5770a4d57640b438f"}
Sep 15 11:51:19.201949 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:19.201909    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:19.912724 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:19.912685    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:19.941078 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:19.941057    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4"
Sep 15 11:51:20.039858 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:20.039766    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/908fc30e-29b6-4090-9e55-66612d32bf59-maas-api-service-ca\") pod \"908fc30e-29b6-4090-9e55-66612d32bf59\" (UID: \"908fc30e-29b6-4090-9e55-66612d32bf59\") "
Sep 15 11:51:20.040105 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:20.040083    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/908fc30e-29b6-4090-9e55-66612d32bf59-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "908fc30e-29b6-4090-9e55-66612d32bf59" (UID: "908fc30e-29b6-4090-9e55-66612d32bf59"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:51:20.141654 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:20.141608    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/908fc30e-29b6-4090-9e55-66612d32bf59-maas-api-service-ca\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:20.566463 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:20.566422    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:20.790285 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:20.790247    2560 generic.go:356] "Generic (PLEG): container finished" podID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" containerID="68884d0e87f30dc64403ca34329de406f8b6f969c27366fe540a37bda8242e62" exitCode=0
Sep 15 11:51:20.790492 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:20.790329    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8" event={"ID":"49fbd7b3-8b22-415d-8f2f-243ea054e36f","Type":"ContainerDied","Data":"68884d0e87f30dc64403ca34329de406f8b6f969c27366fe540a37bda8242e62"}
Sep 15 11:51:20.791288 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:20.791264    2560 scope.go:117] "RemoveContainer" containerID="68884d0e87f30dc64403ca34329de406f8b6f969c27366fe540a37bda8242e62"
Sep 15 11:51:20.792196 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:20.792058    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4" event={"ID":"908fc30e-29b6-4090-9e55-66612d32bf59","Type":"ContainerDied","Data":"9c1d0ca617e835ee81311353d4d3fe19c8209d56cf265c2ab83e65be083383b5"}
Sep 15 11:51:20.792196 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:20.792114    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c1d0ca617e835ee81311353d4d3fe19c8209d56cf265c2ab83e65be083383b5"
Sep 15 11:51:20.792196 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:20.792181    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4"
Sep 15 11:51:20.792852 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:20.792818    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:51:20.792931 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:20.792909    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:51:20.792984 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:20.792953    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:51:21.505590 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:21.505537    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:21.809059 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:21.808955    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8" event={"ID":"49fbd7b3-8b22-415d-8f2f-243ea054e36f","Type":"ContainerStarted","Data":"5309e9280896171824ee16191d32f2be8c0c233d21de1e2d082396a8d7249318"}
Sep 15 11:51:21.992138 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:21.992096    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"]
Sep 15 11:51:22.078134 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:22.078048    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:22.304481 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:22.304442    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:22.649023 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:22.648986    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:51:22.650524 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:22.650491    2560 patch_prober.go:28] interesting pod/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 15 11:51:22.650663 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:22.650554    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8" podUID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 15 11:51:23.177258 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:23.177215    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:23.648796 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:23.648754    2560 patch_prober.go:28] interesting pod/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 15 11:51:23.649008 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:23.648813    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8" podUID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 15 11:51:24.006196 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:24.006111    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:24.648755 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:24.648712    2560 patch_prober.go:28] interesting pod/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 15 11:51:24.648930 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:24.648779    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8" podUID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 15 11:51:24.770149 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:24.770113    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:25.505823 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:25.504861    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:51:25.648848 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:25.648810    2560 patch_prober.go:28] interesting pod/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 15 11:51:25.648998 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:25.648872    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8" podUID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 15 11:51:26.522488 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:26.522452    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:26.648546 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:26.648508    2560 patch_prober.go:28] interesting pod/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 15 11:51:26.648761 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:26.648587    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8" podUID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 15 11:51:27.283608 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:27.283538    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:27.653581 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:27.653538    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:51:27.654492 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:27.654472    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:51:28.150475 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:28.150437    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:29.073320 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:29.073288    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:29.758097 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:29.758061    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:30.409154 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:30.409122    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:30.588251 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:30.588214    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:31.314661 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:31.314625    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:32.284573 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:32.284538    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:33.080870 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.080830    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:33.176886 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.176852    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:33.281980 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.281952    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:33.325109 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.325077    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4"]
Sep 15 11:51:33.331412 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.331329    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-6e996c-q64d4"]
Sep 15 11:51:33.378091 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.378060    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:33.437295 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.437256    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p"]
Sep 15 11:51:33.438665 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.438280    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="908fc30e-29b6-4090-9e55-66612d32bf59" containerName="revoke-keys"
Sep 15 11:51:33.439006 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.438990    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="908fc30e-29b6-4090-9e55-66612d32bf59" containerName="revoke-keys"
Sep 15 11:51:33.439303 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.439284    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="908fc30e-29b6-4090-9e55-66612d32bf59" containerName="revoke-keys"
Sep 15 11:51:33.468843 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.468804    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p"]
Sep 15 11:51:33.469182 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.469161    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p"
Sep 15 11:51:33.483326 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.483299    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbxjk\""
Sep 15 11:51:33.519828 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.519795    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ac49b85-93af-4d4d-b990-a48ad5cccc7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p\" (UID: \"7ac49b85-93af-4d4d-b990-a48ad5cccc7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p"
Sep 15 11:51:33.621404 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.621364    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ac49b85-93af-4d4d-b990-a48ad5cccc7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p\" (UID: \"7ac49b85-93af-4d4d-b990-a48ad5cccc7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p"
Sep 15 11:51:33.622867 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.622843    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ac49b85-93af-4d4d-b990-a48ad5cccc7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p\" (UID: \"7ac49b85-93af-4d4d-b990-a48ad5cccc7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p"
Sep 15 11:51:33.676195 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.676166    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:33.824113 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:33.824081    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p"
Sep 15 11:51:34.009583 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:34.009194    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:34.175075 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:34.175047    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p"]
Sep 15 11:51:34.176551 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:51:34.176522    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac49b85_93af_4d4d_b990_a48ad5cccc7c.slice/crio-4d1a6e3a98e31f36d2f922c31417404d61589685d491e748f1b2bb103f1ffce7 WatchSource:0}: Error finding container 4d1a6e3a98e31f36d2f922c31417404d61589685d491e748f1b2bb103f1ffce7: Status 404 returned error can't find the container with id 4d1a6e3a98e31f36d2f922c31417404d61589685d491e748f1b2bb103f1ffce7
Sep 15 11:51:34.845695 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:34.845662    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:34.921151 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:34.921050    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p" event={"ID":"7ac49b85-93af-4d4d-b990-a48ad5cccc7c","Type":"ContainerStarted","Data":"897079ade6181e918738b2292ef1512f1c48be62a04f01cc7855927e96e18b30"}
Sep 15 11:51:34.921151 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:34.921117    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p" event={"ID":"7ac49b85-93af-4d4d-b990-a48ad5cccc7c","Type":"ContainerStarted","Data":"4d1a6e3a98e31f36d2f922c31417404d61589685d491e748f1b2bb103f1ffce7"}
Sep 15 11:51:34.946820 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:34.946762    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p" podStartSLOduration=1.9467453209999999 podStartE2EDuration="1.946745321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:51:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:51:34.944165971 +0000 UTC m=+1596.436943121" watchObservedRunningTime="2026-09-15 11:51:34.946745321 +0000 UTC m=+1596.439522469"
Sep 15 11:51:35.035847 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:35.035805    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="908fc30e-29b6-4090-9e55-66612d32bf59" path="/var/lib/kubelet/pods/908fc30e-29b6-4090-9e55-66612d32bf59/volumes"
Sep 15 11:51:35.929004 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:35.928967    2560 generic.go:356] "Generic (PLEG): container finished" podID="7ac49b85-93af-4d4d-b990-a48ad5cccc7c" containerID="897079ade6181e918738b2292ef1512f1c48be62a04f01cc7855927e96e18b30" exitCode=0
Sep 15 11:51:35.929413 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:35.929063    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p" event={"ID":"7ac49b85-93af-4d4d-b990-a48ad5cccc7c","Type":"ContainerDied","Data":"897079ade6181e918738b2292ef1512f1c48be62a04f01cc7855927e96e18b30"}
Sep 15 11:51:36.017391 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:36.017354    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:36.821025 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:36.820991    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:37.038350 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.038295    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm" podUID="e7186647-3a95-4239-a909-c02777b107c7" containerName="main" containerID="cri-o://97a1959c92103bea162cf99b6762e4015ef2198c64c6bcbcdff0a3fce300a8b5" gracePeriod=45
Sep 15 11:51:37.166749 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.166727    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p"
Sep 15 11:51:37.265274 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.265243    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ac49b85-93af-4d4d-b990-a48ad5cccc7c-maas-api-service-ca\") pod \"7ac49b85-93af-4d4d-b990-a48ad5cccc7c\" (UID: \"7ac49b85-93af-4d4d-b990-a48ad5cccc7c\") "
Sep 15 11:51:37.265661 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.265633    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ac49b85-93af-4d4d-b990-a48ad5cccc7c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ac49b85-93af-4d4d-b990-a48ad5cccc7c" (UID: "7ac49b85-93af-4d4d-b990-a48ad5cccc7c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:51:37.366350 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.366317    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ac49b85-93af-4d4d-b990-a48ad5cccc7c-maas-api-service-ca\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:37.368505 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.368417    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:51:37.467804 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.467721    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7186647-3a95-4239-a909-c02777b107c7-tls-certs\") pod \"e7186647-3a95-4239-a909-c02777b107c7\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") "
Sep 15 11:51:37.467954 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.467866    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-tmp-dir\") pod \"e7186647-3a95-4239-a909-c02777b107c7\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") "
Sep 15 11:51:37.467954 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.467922    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-model-cache\") pod \"e7186647-3a95-4239-a909-c02777b107c7\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") "
Sep 15 11:51:37.467954 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.467946    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsxc2\" (UniqueName: \"kubernetes.io/projected/e7186647-3a95-4239-a909-c02777b107c7-kube-api-access-lsxc2\") pod \"e7186647-3a95-4239-a909-c02777b107c7\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") "
Sep 15 11:51:37.468128 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.468003    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-home\") pod \"e7186647-3a95-4239-a909-c02777b107c7\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") "
Sep 15 11:51:37.468128 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.468049    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-dshm\") pod \"e7186647-3a95-4239-a909-c02777b107c7\" (UID: \"e7186647-3a95-4239-a909-c02777b107c7\") "
Sep 15 11:51:37.468233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.468176    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-model-cache" (OuterVolumeSpecName: "model-cache") pod "e7186647-3a95-4239-a909-c02777b107c7" (UID: "e7186647-3a95-4239-a909-c02777b107c7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:51:37.468233 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.468186    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e7186647-3a95-4239-a909-c02777b107c7" (UID: "e7186647-3a95-4239-a909-c02777b107c7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:51:37.468337 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.468242    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-home" (OuterVolumeSpecName: "home") pod "e7186647-3a95-4239-a909-c02777b107c7" (UID: "e7186647-3a95-4239-a909-c02777b107c7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:51:37.468510 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.468489    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-tmp-dir\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:37.468592 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.468516    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-model-cache\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:37.468592 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.468529    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-home\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:37.470156 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.470125    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-dshm" (OuterVolumeSpecName: "dshm") pod "e7186647-3a95-4239-a909-c02777b107c7" (UID: "e7186647-3a95-4239-a909-c02777b107c7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:51:37.470266 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.470176    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7186647-3a95-4239-a909-c02777b107c7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e7186647-3a95-4239-a909-c02777b107c7" (UID: "e7186647-3a95-4239-a909-c02777b107c7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:51:37.470266 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.470194    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7186647-3a95-4239-a909-c02777b107c7-kube-api-access-lsxc2" (OuterVolumeSpecName: "kube-api-access-lsxc2") pod "e7186647-3a95-4239-a909-c02777b107c7" (UID: "e7186647-3a95-4239-a909-c02777b107c7"). InnerVolumeSpecName "kube-api-access-lsxc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:51:37.569809 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.569772    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsxc2\" (UniqueName: \"kubernetes.io/projected/e7186647-3a95-4239-a909-c02777b107c7-kube-api-access-lsxc2\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:37.569809 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.569803    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7186647-3a95-4239-a909-c02777b107c7-dshm\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:37.569809 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.569816    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7186647-3a95-4239-a909-c02777b107c7-tls-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:37.892685 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.892587    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:37.945339 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.945308    2560 generic.go:356] "Generic (PLEG): container finished" podID="e7186647-3a95-4239-a909-c02777b107c7" containerID="97a1959c92103bea162cf99b6762e4015ef2198c64c6bcbcdff0a3fce300a8b5" exitCode=0
Sep 15 11:51:37.945515 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.945347    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"
Sep 15 11:51:37.945515 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.945389    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm" event={"ID":"e7186647-3a95-4239-a909-c02777b107c7","Type":"ContainerDied","Data":"97a1959c92103bea162cf99b6762e4015ef2198c64c6bcbcdff0a3fce300a8b5"}
Sep 15 11:51:37.945515 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.945426    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm" event={"ID":"e7186647-3a95-4239-a909-c02777b107c7","Type":"ContainerDied","Data":"15f9563823e63f67aadc9ba7a50853147de1faa4012312ca8234ac483de07b85"}
Sep 15 11:51:37.945515 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.945445    2560 scope.go:117] "RemoveContainer" containerID="97a1959c92103bea162cf99b6762e4015ef2198c64c6bcbcdff0a3fce300a8b5"
Sep 15 11:51:37.946918 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.946897    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p" event={"ID":"7ac49b85-93af-4d4d-b990-a48ad5cccc7c","Type":"ContainerDied","Data":"4d1a6e3a98e31f36d2f922c31417404d61589685d491e748f1b2bb103f1ffce7"}
Sep 15 11:51:37.946987 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.946924    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p"
Sep 15 11:51:37.946987 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.946943    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d1a6e3a98e31f36d2f922c31417404d61589685d491e748f1b2bb103f1ffce7"
Sep 15 11:51:37.970691 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.970661    2560 scope.go:117] "RemoveContainer" containerID="97a1959c92103bea162cf99b6762e4015ef2198c64c6bcbcdff0a3fce300a8b5"
Sep 15 11:51:37.971006 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:51:37.970986    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97a1959c92103bea162cf99b6762e4015ef2198c64c6bcbcdff0a3fce300a8b5\": container with ID starting with 97a1959c92103bea162cf99b6762e4015ef2198c64c6bcbcdff0a3fce300a8b5 not found: ID does not exist" containerID="97a1959c92103bea162cf99b6762e4015ef2198c64c6bcbcdff0a3fce300a8b5"
Sep 15 11:51:37.971090 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.971014    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a1959c92103bea162cf99b6762e4015ef2198c64c6bcbcdff0a3fce300a8b5"} err="failed to get container status \"97a1959c92103bea162cf99b6762e4015ef2198c64c6bcbcdff0a3fce300a8b5\": rpc error: code = NotFound desc = could not find container \"97a1959c92103bea162cf99b6762e4015ef2198c64c6bcbcdff0a3fce300a8b5\": container with ID starting with 97a1959c92103bea162cf99b6762e4015ef2198c64c6bcbcdff0a3fce300a8b5 not found: ID does not exist"
Sep 15 11:51:38.003585 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:37.988548    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"]
Sep 15 11:51:38.003585 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:38.000185    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-6e996c/embedding-w0-6e996c-kserve-6f874cd68-jllhm"]
Sep 15 11:51:38.846988 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:38.846945    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:39.036907 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:39.036873    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7186647-3a95-4239-a909-c02777b107c7" path="/var/lib/kubelet/pods/e7186647-3a95-4239-a909-c02777b107c7/volumes"
Sep 15 11:51:39.476622 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:39.476593    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:40.072224 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:40.072144    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:40.148273 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:40.148183    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:40.333495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:40.333405    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:40.381629 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:40.381585    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"]
Sep 15 11:51:40.381956 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:40.381911    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz" podUID="8f737c66-5417-491d-8c79-9a4649e2eb7a" containerName="istio-proxy" containerID="cri-o://49790447d4ebe38548c046d8419ce2c636f7051005e7fd9bd2a99583ee909d77" gracePeriod=30
Sep 15 11:51:40.483696 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:40.475763    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:41.927503 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:41.927465    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:42.980399 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:42.980367    2560 generic.go:356] "Generic (PLEG): container finished" podID="152bfca4-485b-4870-92fb-c41e53ed1e3d" containerID="4185c467f67c2335e39894010b64c78275ab539d9d2a8baca563c7868e391526" exitCode=0
Sep 15 11:51:42.980838 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:42.980422    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw" event={"ID":"152bfca4-485b-4870-92fb-c41e53ed1e3d","Type":"ContainerDied","Data":"4185c467f67c2335e39894010b64c78275ab539d9d2a8baca563c7868e391526"}
Sep 15 11:51:42.980838 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:42.980833    2560 scope.go:117] "RemoveContainer" containerID="4185c467f67c2335e39894010b64c78275ab539d9d2a8baca563c7868e391526"
Sep 15 11:51:42.982001 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:42.981971    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:51:42.982064 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:42.982040    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:51:42.982102 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:42.982069    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:51:43.990202 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:43.989348    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw" event={"ID":"152bfca4-485b-4870-92fb-c41e53ed1e3d","Type":"ContainerStarted","Data":"e1c6707dd3e6ec1e6dede3526f78eedfad18dc0a687ce70d0645905deb625ca4"}
Sep 15 11:51:44.553029 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:44.552993    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:44.554239 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:44.554216    2560 patch_prober.go:28] interesting pod/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Sep 15 11:51:44.554302 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:44.554264    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw" podUID="152bfca4-485b-4870-92fb-c41e53ed1e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Sep 15 11:51:44.741316 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:44.741283    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:45.553136 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.553099    2560 patch_prober.go:28] interesting pod/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Sep 15 11:51:45.553597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.553549    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw" podUID="152bfca4-485b-4870-92fb-c41e53ed1e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Sep 15 11:51:45.668190 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.668132    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:51:45.759087 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759056    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czs7d\" (UniqueName: \"kubernetes.io/projected/8f737c66-5417-491d-8c79-9a4649e2eb7a-kube-api-access-czs7d\") pod \"8f737c66-5417-491d-8c79-9a4649e2eb7a\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") "
Sep 15 11:51:45.759258 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759107    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-envoy\") pod \"8f737c66-5417-491d-8c79-9a4649e2eb7a\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") "
Sep 15 11:51:45.759258 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759138    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-data\") pod \"8f737c66-5417-491d-8c79-9a4649e2eb7a\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") "
Sep 15 11:51:45.759258 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759205    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f737c66-5417-491d-8c79-9a4649e2eb7a-istiod-ca-cert\") pod \"8f737c66-5417-491d-8c79-9a4649e2eb7a\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") "
Sep 15 11:51:45.759258 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759234    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-credential-socket\") pod \"8f737c66-5417-491d-8c79-9a4649e2eb7a\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") "
Sep 15 11:51:45.759493 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759268    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-workload-socket\") pod \"8f737c66-5417-491d-8c79-9a4649e2eb7a\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") "
Sep 15 11:51:45.759493 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759290    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-podinfo\") pod \"8f737c66-5417-491d-8c79-9a4649e2eb7a\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") "
Sep 15 11:51:45.759493 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759346    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-workload-certs\") pod \"8f737c66-5417-491d-8c79-9a4649e2eb7a\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") "
Sep 15 11:51:45.759493 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759394    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-token\") pod \"8f737c66-5417-491d-8c79-9a4649e2eb7a\" (UID: \"8f737c66-5417-491d-8c79-9a4649e2eb7a\") "
Sep 15 11:51:45.759493 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759466    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f737c66-5417-491d-8c79-9a4649e2eb7a" (UID: "8f737c66-5417-491d-8c79-9a4649e2eb7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:51:45.759776 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759602    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f737c66-5417-491d-8c79-9a4649e2eb7a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f737c66-5417-491d-8c79-9a4649e2eb7a" (UID: "8f737c66-5417-491d-8c79-9a4649e2eb7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:51:45.759776 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759615    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f737c66-5417-491d-8c79-9a4649e2eb7a" (UID: "8f737c66-5417-491d-8c79-9a4649e2eb7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:51:45.759776 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759718    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f737c66-5417-491d-8c79-9a4649e2eb7a" (UID: "8f737c66-5417-491d-8c79-9a4649e2eb7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:51:45.759776 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759740    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f737c66-5417-491d-8c79-9a4649e2eb7a" (UID: "8f737c66-5417-491d-8c79-9a4649e2eb7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:51:45.759943 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759885    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:45.759943 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759905    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:45.759943 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759918    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f737c66-5417-491d-8c79-9a4649e2eb7a-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:45.759943 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759930    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:45.760082 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.759945    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:45.761524 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.761502    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f737c66-5417-491d-8c79-9a4649e2eb7a" (UID: "8f737c66-5417-491d-8c79-9a4649e2eb7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:51:45.761641 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.761556    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f737c66-5417-491d-8c79-9a4649e2eb7a" (UID: "8f737c66-5417-491d-8c79-9a4649e2eb7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:51:45.761698 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.761650    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f737c66-5417-491d-8c79-9a4649e2eb7a-kube-api-access-czs7d" (OuterVolumeSpecName: "kube-api-access-czs7d") pod "8f737c66-5417-491d-8c79-9a4649e2eb7a" (UID: "8f737c66-5417-491d-8c79-9a4649e2eb7a"). InnerVolumeSpecName "kube-api-access-czs7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:51:45.761952 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.761925    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f737c66-5417-491d-8c79-9a4649e2eb7a" (UID: "8f737c66-5417-491d-8c79-9a4649e2eb7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:51:45.861112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.861077    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:45.861112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.861107    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czs7d\" (UniqueName: \"kubernetes.io/projected/8f737c66-5417-491d-8c79-9a4649e2eb7a-kube-api-access-czs7d\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:45.861112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.861116    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:45.861353 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:45.861125    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f737c66-5417-491d-8c79-9a4649e2eb7a-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:46.013288 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:46.012326    2560 generic.go:356] "Generic (PLEG): container finished" podID="8f737c66-5417-491d-8c79-9a4649e2eb7a" containerID="49790447d4ebe38548c046d8419ce2c636f7051005e7fd9bd2a99583ee909d77" exitCode=0
Sep 15 11:51:46.013288 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:46.012403    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz" event={"ID":"8f737c66-5417-491d-8c79-9a4649e2eb7a","Type":"ContainerDied","Data":"49790447d4ebe38548c046d8419ce2c636f7051005e7fd9bd2a99583ee909d77"}
Sep 15 11:51:46.013288 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:46.012499    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz" event={"ID":"8f737c66-5417-491d-8c79-9a4649e2eb7a","Type":"ContainerDied","Data":"d5a23d981f2f6f33c9c735bb33de3c6bd9cbc4f6197b330d09c062c442e779f3"}
Sep 15 11:51:46.013288 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:46.012532    2560 scope.go:117] "RemoveContainer" containerID="49790447d4ebe38548c046d8419ce2c636f7051005e7fd9bd2a99583ee909d77"
Sep 15 11:51:46.013659 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:46.013377    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"
Sep 15 11:51:46.024502 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:46.024485    2560 scope.go:117] "RemoveContainer" containerID="023166f619ab846f2d9d82fbc15449a5a5fd741cb302ca251c60e1de04c7b612"
Sep 15 11:51:46.033256 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:46.033238    2560 scope.go:117] "RemoveContainer" containerID="49790447d4ebe38548c046d8419ce2c636f7051005e7fd9bd2a99583ee909d77"
Sep 15 11:51:46.033510 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:51:46.033492    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49790447d4ebe38548c046d8419ce2c636f7051005e7fd9bd2a99583ee909d77\": container with ID starting with 49790447d4ebe38548c046d8419ce2c636f7051005e7fd9bd2a99583ee909d77 not found: ID does not exist" containerID="49790447d4ebe38548c046d8419ce2c636f7051005e7fd9bd2a99583ee909d77"
Sep 15 11:51:46.033597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:46.033517    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49790447d4ebe38548c046d8419ce2c636f7051005e7fd9bd2a99583ee909d77"} err="failed to get container status \"49790447d4ebe38548c046d8419ce2c636f7051005e7fd9bd2a99583ee909d77\": rpc error: code = NotFound desc = could not find container \"49790447d4ebe38548c046d8419ce2c636f7051005e7fd9bd2a99583ee909d77\": container with ID starting with 49790447d4ebe38548c046d8419ce2c636f7051005e7fd9bd2a99583ee909d77 not found: ID does not exist"
Sep 15 11:51:46.033597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:46.033533    2560 scope.go:117] "RemoveContainer" containerID="023166f619ab846f2d9d82fbc15449a5a5fd741cb302ca251c60e1de04c7b612"
Sep 15 11:51:46.033929 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:51:46.033911    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"023166f619ab846f2d9d82fbc15449a5a5fd741cb302ca251c60e1de04c7b612\": container with ID starting with 023166f619ab846f2d9d82fbc15449a5a5fd741cb302ca251c60e1de04c7b612 not found: ID does not exist" containerID="023166f619ab846f2d9d82fbc15449a5a5fd741cb302ca251c60e1de04c7b612"
Sep 15 11:51:46.033999 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:46.033936    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"023166f619ab846f2d9d82fbc15449a5a5fd741cb302ca251c60e1de04c7b612"} err="failed to get container status \"023166f619ab846f2d9d82fbc15449a5a5fd741cb302ca251c60e1de04c7b612\": rpc error: code = NotFound desc = could not find container \"023166f619ab846f2d9d82fbc15449a5a5fd741cb302ca251c60e1de04c7b612\": container with ID starting with 023166f619ab846f2d9d82fbc15449a5a5fd741cb302ca251c60e1de04c7b612 not found: ID does not exist"
Sep 15 11:51:46.038801 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:46.038779    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"]
Sep 15 11:51:46.045083 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:46.045060    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-6e996c-openshift-default-575d5b56b8-k88zz"]
Sep 15 11:51:46.552994 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:46.552956    2560 patch_prober.go:28] interesting pod/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Sep 15 11:51:46.553172 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:46.553036    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw" podUID="152bfca4-485b-4870-92fb-c41e53ed1e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Sep 15 11:51:47.035335 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:47.035298    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f737c66-5417-491d-8c79-9a4649e2eb7a" path="/var/lib/kubelet/pods/8f737c66-5417-491d-8c79-9a4649e2eb7a/volumes"
Sep 15 11:51:47.553001 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:47.552963    2560 patch_prober.go:28] interesting pod/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Sep 15 11:51:47.553200 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:47.553021    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw" podUID="152bfca4-485b-4870-92fb-c41e53ed1e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Sep 15 11:51:48.553384 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:48.553350    2560 patch_prober.go:28] interesting pod/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Sep 15 11:51:48.553809 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:48.553404    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw" podUID="152bfca4-485b-4870-92fb-c41e53ed1e3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Sep 15 11:51:49.123593 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.123540    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:49.161912 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.161881    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"]
Sep 15 11:51:49.162138 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.162109    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd" podUID="5ce9202b-2e85-448a-9811-d211f5da58a2" containerName="payload-processing" containerID="cri-o://01101d4322bd8783238d38384ebc5f0dce7766a720f2d9c5c4ac7e4db0c706e1" gracePeriod=30
Sep 15 11:51:49.171043 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.170997    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"]
Sep 15 11:51:49.171202 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.171169    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh" podUID="772effbf-c80e-4501-97ac-5fa9cdb067a5" containerName="payload-pre-processing" containerID="cri-o://8cc3a61325a37d21b2919132ece3a985f717992ac277408618e91c672505878f" gracePeriod=30
Sep 15 11:51:49.219405 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.219358    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:49.285002 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.284975    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:49.366676 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.364572    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p"]
Sep 15 11:51:49.369112 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.369076    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-bcb732-l9b6p"]
Sep 15 11:51:49.384708 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.384549    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:49.455730 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.455693    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"
Sep 15 11:51:49.498639 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.494879    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:49.548469 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.548435    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"
Sep 15 11:51:49.559623 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.559590    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:49.560999 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.560976    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:51:49.604697 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.603251    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q89ln\" (UniqueName: \"kubernetes.io/projected/5ce9202b-2e85-448a-9811-d211f5da58a2-kube-api-access-q89ln\") pod \"5ce9202b-2e85-448a-9811-d211f5da58a2\" (UID: \"5ce9202b-2e85-448a-9811-d211f5da58a2\") "
Sep 15 11:51:49.604697 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.603317    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ce9202b-2e85-448a-9811-d211f5da58a2-plugins-config-volume\") pod \"5ce9202b-2e85-448a-9811-d211f5da58a2\" (UID: \"5ce9202b-2e85-448a-9811-d211f5da58a2\") "
Sep 15 11:51:49.604697 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.604458    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ce9202b-2e85-448a-9811-d211f5da58a2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5ce9202b-2e85-448a-9811-d211f5da58a2" (UID: "5ce9202b-2e85-448a-9811-d211f5da58a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:51:49.608805 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.608636    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ce9202b-2e85-448a-9811-d211f5da58a2-kube-api-access-q89ln" (OuterVolumeSpecName: "kube-api-access-q89ln") pod "5ce9202b-2e85-448a-9811-d211f5da58a2" (UID: "5ce9202b-2e85-448a-9811-d211f5da58a2"). InnerVolumeSpecName "kube-api-access-q89ln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:51:49.702188 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.702095    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:49.704732 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.704703    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hnxb\" (UniqueName: \"kubernetes.io/projected/772effbf-c80e-4501-97ac-5fa9cdb067a5-kube-api-access-4hnxb\") pod \"772effbf-c80e-4501-97ac-5fa9cdb067a5\" (UID: \"772effbf-c80e-4501-97ac-5fa9cdb067a5\") "
Sep 15 11:51:49.704887 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.704863    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/772effbf-c80e-4501-97ac-5fa9cdb067a5-plugins-config-volume\") pod \"772effbf-c80e-4501-97ac-5fa9cdb067a5\" (UID: \"772effbf-c80e-4501-97ac-5fa9cdb067a5\") "
Sep 15 11:51:49.705282 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.705249    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/772effbf-c80e-4501-97ac-5fa9cdb067a5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "772effbf-c80e-4501-97ac-5fa9cdb067a5" (UID: "772effbf-c80e-4501-97ac-5fa9cdb067a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:51:49.705408 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.705307    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q89ln\" (UniqueName: \"kubernetes.io/projected/5ce9202b-2e85-448a-9811-d211f5da58a2-kube-api-access-q89ln\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:49.705408 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.705330    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ce9202b-2e85-448a-9811-d211f5da58a2-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:49.707086 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.707061    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/772effbf-c80e-4501-97ac-5fa9cdb067a5-kube-api-access-4hnxb" (OuterVolumeSpecName: "kube-api-access-4hnxb") pod "772effbf-c80e-4501-97ac-5fa9cdb067a5" (UID: "772effbf-c80e-4501-97ac-5fa9cdb067a5"). InnerVolumeSpecName "kube-api-access-4hnxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:51:49.806085 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.806041    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hnxb\" (UniqueName: \"kubernetes.io/projected/772effbf-c80e-4501-97ac-5fa9cdb067a5-kube-api-access-4hnxb\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:49.806085 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.806088    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/772effbf-c80e-4501-97ac-5fa9cdb067a5-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:51:49.865360 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:49.865325    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:50.040452 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.040368    2560 generic.go:356] "Generic (PLEG): container finished" podID="5ce9202b-2e85-448a-9811-d211f5da58a2" containerID="01101d4322bd8783238d38384ebc5f0dce7766a720f2d9c5c4ac7e4db0c706e1" exitCode=0
Sep 15 11:51:50.040452 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.040407    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"
Sep 15 11:51:50.040693 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.040446    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd" event={"ID":"5ce9202b-2e85-448a-9811-d211f5da58a2","Type":"ContainerDied","Data":"01101d4322bd8783238d38384ebc5f0dce7766a720f2d9c5c4ac7e4db0c706e1"}
Sep 15 11:51:50.040693 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.040486    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd" event={"ID":"5ce9202b-2e85-448a-9811-d211f5da58a2","Type":"ContainerDied","Data":"d4382d970312a1a9867e66680e4656ced14950b408d9ad9712ac76f783fc54ab"}
Sep 15 11:51:50.040693 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.040509    2560 scope.go:117] "RemoveContainer" containerID="01101d4322bd8783238d38384ebc5f0dce7766a720f2d9c5c4ac7e4db0c706e1"
Sep 15 11:51:50.041890 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.041865    2560 generic.go:356] "Generic (PLEG): container finished" podID="772effbf-c80e-4501-97ac-5fa9cdb067a5" containerID="8cc3a61325a37d21b2919132ece3a985f717992ac277408618e91c672505878f" exitCode=0
Sep 15 11:51:50.041990 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.041917    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"
Sep 15 11:51:50.041990 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.041951    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh" event={"ID":"772effbf-c80e-4501-97ac-5fa9cdb067a5","Type":"ContainerDied","Data":"8cc3a61325a37d21b2919132ece3a985f717992ac277408618e91c672505878f"}
Sep 15 11:51:50.042078 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.042008    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh" event={"ID":"772effbf-c80e-4501-97ac-5fa9cdb067a5","Type":"ContainerDied","Data":"686ec542384d7df549e4025e3edd069b16f6fc99c853d48415c8a8b126255e40"}
Sep 15 11:51:50.053981 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.053967    2560 scope.go:117] "RemoveContainer" containerID="01101d4322bd8783238d38384ebc5f0dce7766a720f2d9c5c4ac7e4db0c706e1"
Sep 15 11:51:50.054246 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:51:50.054228    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01101d4322bd8783238d38384ebc5f0dce7766a720f2d9c5c4ac7e4db0c706e1\": container with ID starting with 01101d4322bd8783238d38384ebc5f0dce7766a720f2d9c5c4ac7e4db0c706e1 not found: ID does not exist" containerID="01101d4322bd8783238d38384ebc5f0dce7766a720f2d9c5c4ac7e4db0c706e1"
Sep 15 11:51:50.054314 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.054251    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01101d4322bd8783238d38384ebc5f0dce7766a720f2d9c5c4ac7e4db0c706e1"} err="failed to get container status \"01101d4322bd8783238d38384ebc5f0dce7766a720f2d9c5c4ac7e4db0c706e1\": rpc error: code = NotFound desc = could not find container \"01101d4322bd8783238d38384ebc5f0dce7766a720f2d9c5c4ac7e4db0c706e1\": container with ID starting with 01101d4322bd8783238d38384ebc5f0dce7766a720f2d9c5c4ac7e4db0c706e1 not found: ID does not exist"
Sep 15 11:51:50.054314 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.054267    2560 scope.go:117] "RemoveContainer" containerID="8cc3a61325a37d21b2919132ece3a985f717992ac277408618e91c672505878f"
Sep 15 11:51:50.063135 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.063114    2560 scope.go:117] "RemoveContainer" containerID="8cc3a61325a37d21b2919132ece3a985f717992ac277408618e91c672505878f"
Sep 15 11:51:50.063419 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:51:50.063398    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cc3a61325a37d21b2919132ece3a985f717992ac277408618e91c672505878f\": container with ID starting with 8cc3a61325a37d21b2919132ece3a985f717992ac277408618e91c672505878f not found: ID does not exist" containerID="8cc3a61325a37d21b2919132ece3a985f717992ac277408618e91c672505878f"
Sep 15 11:51:50.063518 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.063422    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc3a61325a37d21b2919132ece3a985f717992ac277408618e91c672505878f"} err="failed to get container status \"8cc3a61325a37d21b2919132ece3a985f717992ac277408618e91c672505878f\": rpc error: code = NotFound desc = could not find container \"8cc3a61325a37d21b2919132ece3a985f717992ac277408618e91c672505878f\": container with ID starting with 8cc3a61325a37d21b2919132ece3a985f717992ac277408618e91c672505878f not found: ID does not exist"
Sep 15 11:51:50.065280 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.065262    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"]
Sep 15 11:51:50.071232 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.071212    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-bcb732-778b8b4b5c-zdmzd"]
Sep 15 11:51:50.081752 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.081730    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"]
Sep 15 11:51:50.085605 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.085556    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-bcb732-56896f7658-zvfgh"]
Sep 15 11:51:50.621197 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:50.621164    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:51.036965 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:51.036884    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ce9202b-2e85-448a-9811-d211f5da58a2" path="/var/lib/kubelet/pods/5ce9202b-2e85-448a-9811-d211f5da58a2/volumes"
Sep 15 11:51:51.037394 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:51.037376    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="772effbf-c80e-4501-97ac-5fa9cdb067a5" path="/var/lib/kubelet/pods/772effbf-c80e-4501-97ac-5fa9cdb067a5/volumes"
Sep 15 11:51:51.037728 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:51.037715    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ac49b85-93af-4d4d-b990-a48ad5cccc7c" path="/var/lib/kubelet/pods/7ac49b85-93af-4d4d-b990-a48ad5cccc7c/volumes"
Sep 15 11:51:51.304582 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:51.304454    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:52.279792 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:52.279754    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:52.928708 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:52.928673    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:53.684754 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:53.684716    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:54.378819 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:54.378777    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:55.696412 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:55.696379    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:55.760966 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:55.760919    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:55.929451 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:55.929416    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"]
Sep 15 11:51:55.929663 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:55.929624    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8" podUID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" containerName="istio-proxy" containerID="cri-o://5309e9280896171824ee16191d32f2be8c0c233d21de1e2d082396a8d7249318" gracePeriod=30
Sep 15 11:51:55.946318 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:55.946287    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:56.580609 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:56.580553    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:57.303467 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:57.303430    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:57.655699 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:57.655665    2560 patch_prober.go:28] interesting pod/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 11:51:57.655876 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:57.655735    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8" podUID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 11:51:57.985247 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:57.985165    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:58.697641 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:58.697611    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:59.520308 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:59.520267    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:59.621983 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:59.621948    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:59.717683 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:59.717648    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:51:59.753654 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:51:59.753616    2560 scope.go:117] "RemoveContainer" containerID="8fbd468f635382b12d39b3d6d4732e12b1e587e03ce54175a47c028440b32f98"
Sep 15 11:52:00.207492 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:00.207453    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:00.400555 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:00.400516    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:01.124195 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.123931    2560 generic.go:356] "Generic (PLEG): container finished" podID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" containerID="5309e9280896171824ee16191d32f2be8c0c233d21de1e2d082396a8d7249318" exitCode=0
Sep 15 11:52:01.124195 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.124026    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8" event={"ID":"49fbd7b3-8b22-415d-8f2f-243ea054e36f","Type":"ContainerDied","Data":"5309e9280896171824ee16191d32f2be8c0c233d21de1e2d082396a8d7249318"}
Sep 15 11:52:01.124195 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.124083    2560 scope.go:117] "RemoveContainer" containerID="68884d0e87f30dc64403ca34329de406f8b6f969c27366fe540a37bda8242e62"
Sep 15 11:52:01.210810 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.210729    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:52:01.327760 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.327728    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-token\") pod \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") "
Sep 15 11:52:01.327925 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.327769    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-workload-certs\") pod \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") "
Sep 15 11:52:01.327977 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.327940    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-credential-socket\") pod \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") "
Sep 15 11:52:01.328045 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.328012    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istiod-ca-cert\") pod \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") "
Sep 15 11:52:01.328105 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.328057    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "49fbd7b3-8b22-415d-8f2f-243ea054e36f" (UID: "49fbd7b3-8b22-415d-8f2f-243ea054e36f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:01.328105 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.328087    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtj62\" (UniqueName: \"kubernetes.io/projected/49fbd7b3-8b22-415d-8f2f-243ea054e36f-kube-api-access-wtj62\") pod \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") "
Sep 15 11:52:01.328217 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.328158    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-podinfo\") pod \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") "
Sep 15 11:52:01.328217 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.328173    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "49fbd7b3-8b22-415d-8f2f-243ea054e36f" (UID: "49fbd7b3-8b22-415d-8f2f-243ea054e36f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:01.328332 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.328235    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-workload-socket\") pod \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") "
Sep 15 11:52:01.328332 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.328264    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-envoy\") pod \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") "
Sep 15 11:52:01.328332 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.328310    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-data\") pod \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\" (UID: \"49fbd7b3-8b22-415d-8f2f-243ea054e36f\") "
Sep 15 11:52:01.328332 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.328315    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "49fbd7b3-8b22-415d-8f2f-243ea054e36f" (UID: "49fbd7b3-8b22-415d-8f2f-243ea054e36f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:52:01.328927 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.328456    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "49fbd7b3-8b22-415d-8f2f-243ea054e36f" (UID: "49fbd7b3-8b22-415d-8f2f-243ea054e36f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:01.328927 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.328723    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:01.328927 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.328745    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:01.328927 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.328759    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:01.328927 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.328771    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:01.328927 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.328769    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-data" (OuterVolumeSpecName: "istio-data") pod "49fbd7b3-8b22-415d-8f2f-243ea054e36f" (UID: "49fbd7b3-8b22-415d-8f2f-243ea054e36f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:01.330841 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.330819    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "49fbd7b3-8b22-415d-8f2f-243ea054e36f" (UID: "49fbd7b3-8b22-415d-8f2f-243ea054e36f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:52:01.330963 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.330944    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "49fbd7b3-8b22-415d-8f2f-243ea054e36f" (UID: "49fbd7b3-8b22-415d-8f2f-243ea054e36f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:01.331554 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.331529    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49fbd7b3-8b22-415d-8f2f-243ea054e36f-kube-api-access-wtj62" (OuterVolumeSpecName: "kube-api-access-wtj62") pod "49fbd7b3-8b22-415d-8f2f-243ea054e36f" (UID: "49fbd7b3-8b22-415d-8f2f-243ea054e36f"). InnerVolumeSpecName "kube-api-access-wtj62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:52:01.331655 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.331591    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-token" (OuterVolumeSpecName: "istio-token") pod "49fbd7b3-8b22-415d-8f2f-243ea054e36f" (UID: "49fbd7b3-8b22-415d-8f2f-243ea054e36f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:52:01.376606 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.376505    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:01.429892 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.429848    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:01.429892 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.429887    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:01.430116 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.429902    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:01.430116 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.429916    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtj62\" (UniqueName: \"kubernetes.io/projected/49fbd7b3-8b22-415d-8f2f-243ea054e36f-kube-api-access-wtj62\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:01.430116 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:01.429928    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49fbd7b3-8b22-415d-8f2f-243ea054e36f-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:02.097510 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:02.097153    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:02.132097 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:02.132068    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"
Sep 15 11:52:02.132478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:02.132071    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8" event={"ID":"49fbd7b3-8b22-415d-8f2f-243ea054e36f","Type":"ContainerDied","Data":"0fa7f75812d4b1d738760669ad94f9c83e3ff306119df3743eafca884dabd892"}
Sep 15 11:52:02.132478 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:02.132202    2560 scope.go:117] "RemoveContainer" containerID="5309e9280896171824ee16191d32f2be8c0c233d21de1e2d082396a8d7249318"
Sep 15 11:52:02.157969 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:02.157940    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"]
Sep 15 11:52:02.162083 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:02.162057    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-bcb732-openshift-default-c4f7cc554-ljkb8"]
Sep 15 11:52:03.035887 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:03.035854    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" path="/var/lib/kubelet/pods/49fbd7b3-8b22-415d-8f2f-243ea054e36f/volumes"
Sep 15 11:52:04.499113 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:04.499074    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:06.481374 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:06.481334    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:06.970734 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:06.970698    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:07.084863 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.084824    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb"]
Sep 15 11:52:07.085416 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085396    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" containerName="istio-proxy"
Sep 15 11:52:07.085416 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085417    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" containerName="istio-proxy"
Sep 15 11:52:07.085597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085427    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ce9202b-2e85-448a-9811-d211f5da58a2" containerName="payload-processing"
Sep 15 11:52:07.085597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085433    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ce9202b-2e85-448a-9811-d211f5da58a2" containerName="payload-processing"
Sep 15 11:52:07.085597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085448    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7186647-3a95-4239-a909-c02777b107c7" containerName="main"
Sep 15 11:52:07.085597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085457    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7186647-3a95-4239-a909-c02777b107c7" containerName="main"
Sep 15 11:52:07.085597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085468    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f737c66-5417-491d-8c79-9a4649e2eb7a" containerName="istio-proxy"
Sep 15 11:52:07.085597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085474    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f737c66-5417-491d-8c79-9a4649e2eb7a" containerName="istio-proxy"
Sep 15 11:52:07.085597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085497    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="772effbf-c80e-4501-97ac-5fa9cdb067a5" containerName="payload-pre-processing"
Sep 15 11:52:07.085597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085503    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="772effbf-c80e-4501-97ac-5fa9cdb067a5" containerName="payload-pre-processing"
Sep 15 11:52:07.085597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085508    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ac49b85-93af-4d4d-b990-a48ad5cccc7c" containerName="revoke-keys"
Sep 15 11:52:07.085597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085513    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ac49b85-93af-4d4d-b990-a48ad5cccc7c" containerName="revoke-keys"
Sep 15 11:52:07.085597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085594    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f737c66-5417-491d-8c79-9a4649e2eb7a" containerName="istio-proxy"
Sep 15 11:52:07.085597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085604    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ac49b85-93af-4d4d-b990-a48ad5cccc7c" containerName="revoke-keys"
Sep 15 11:52:07.085978 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085610    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" containerName="istio-proxy"
Sep 15 11:52:07.085978 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085616    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ce9202b-2e85-448a-9811-d211f5da58a2" containerName="payload-processing"
Sep 15 11:52:07.085978 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085623    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="772effbf-c80e-4501-97ac-5fa9cdb067a5" containerName="payload-pre-processing"
Sep 15 11:52:07.085978 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085629    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7186647-3a95-4239-a909-c02777b107c7" containerName="main"
Sep 15 11:52:07.085978 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.085637    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" containerName="istio-proxy"
Sep 15 11:52:07.090258 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.090224    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb"
Sep 15 11:52:07.092766 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.092742    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbxjk\""
Sep 15 11:52:07.100508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.100463    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb"]
Sep 15 11:52:07.185576 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.185539    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb\" (UID: \"0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb"
Sep 15 11:52:07.286312 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.286225    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb\" (UID: \"0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb"
Sep 15 11:52:07.287050 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.286959    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb\" (UID: \"0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb"
Sep 15 11:52:07.405102 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.405069    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb"
Sep 15 11:52:07.485876 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.485821    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:07.682227 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:07.682203    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb"]
Sep 15 11:52:07.684001 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:52:07.683974    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e9f566e_e9e5_4b85_ab54_d9c2a05ecb9e.slice/crio-a0039cecefaa3199d5cbf77e312173e22b42e7728340dd404fb7b069ad13df37 WatchSource:0}: Error finding container a0039cecefaa3199d5cbf77e312173e22b42e7728340dd404fb7b069ad13df37: Status 404 returned error can't find the container with id a0039cecefaa3199d5cbf77e312173e22b42e7728340dd404fb7b069ad13df37
Sep 15 11:52:08.168794 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:08.168760    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb" event={"ID":"0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e","Type":"ContainerStarted","Data":"89a297066ea32719085db705529e1aa308b61c4a82438e4c4811c234bacc8ea5"}
Sep 15 11:52:08.168963 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:08.168822    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb" event={"ID":"0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e","Type":"ContainerStarted","Data":"a0039cecefaa3199d5cbf77e312173e22b42e7728340dd404fb7b069ad13df37"}
Sep 15 11:52:08.191037 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:08.190957    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb" podStartSLOduration=1.1909389049999999 podStartE2EDuration="1.190938905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:52:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:52:08.190258478 +0000 UTC m=+1629.683035650" watchObservedRunningTime="2026-09-15 11:52:08.190938905 +0000 UTC m=+1629.683716055"
Sep 15 11:52:08.488174 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:08.488137    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:09.149062 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:09.149023    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:09.176495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:09.176461    2560 generic.go:356] "Generic (PLEG): container finished" podID="0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e" containerID="89a297066ea32719085db705529e1aa308b61c4a82438e4c4811c234bacc8ea5" exitCode=0
Sep 15 11:52:09.176698 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:09.176533    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb" event={"ID":"0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e","Type":"ContainerDied","Data":"89a297066ea32719085db705529e1aa308b61c4a82438e4c4811c234bacc8ea5"}
Sep 15 11:52:09.783708 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:09.783669    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:10.327016 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:10.326989    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb"
Sep 15 11:52:10.425712 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:10.425680    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e-maas-api-service-ca\") pod \"0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e\" (UID: \"0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e\") "
Sep 15 11:52:10.426044 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:10.426019    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e" (UID: "0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:52:10.526719 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:10.526640    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e-maas-api-service-ca\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:10.589163 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:10.589118    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:11.189991 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:11.189956    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb" event={"ID":"0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e","Type":"ContainerDied","Data":"a0039cecefaa3199d5cbf77e312173e22b42e7728340dd404fb7b069ad13df37"}
Sep 15 11:52:11.190367 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:11.190009    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0039cecefaa3199d5cbf77e312173e22b42e7728340dd404fb7b069ad13df37"
Sep 15 11:52:11.190367 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:11.189970    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb"
Sep 15 11:52:11.268704 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:11.268649    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:12.032051 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:12.031989    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:12.160840 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:12.160801    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:13.284530 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:13.284496    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:13.359744 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:13.359709    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:13.461556 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:13.461520    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:13.531253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:13.531213    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:13.867935 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:13.867901    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:14.522829 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:14.522774    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:15.630333 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:15.630297    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:16.675495 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:16.675453    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:17.869743 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:17.869704    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:18.699791 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:18.699759    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:20.176537 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:20.176499    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:20.297237 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:20.297199    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:20.402963 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:20.402930    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"]
Sep 15 11:52:20.403198 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:20.403173    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw" podUID="152bfca4-485b-4870-92fb-c41e53ed1e3d" containerName="istio-proxy" containerID="cri-o://e1c6707dd3e6ec1e6dede3526f78eedfad18dc0a687ce70d0645905deb625ca4" gracePeriod=30
Sep 15 11:52:20.419795 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:20.419755    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:21.013871 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:21.013824    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:21.877146 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:21.877110    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:22.568597 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:22.568549    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:23.308734 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.308698    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:23.317229 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.317201    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b8fbdd-29824545-bmjws"]
Sep 15 11:52:23.319552 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.319528    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b8fbdd-29824545-bmjws"]
Sep 15 11:52:23.339271 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.339242    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"]
Sep 15 11:52:23.339512 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.339482    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw" podUID="2233a306-f57a-47aa-8c1f-3d3c5ae642d7" containerName="payload-processing" containerID="cri-o://1f3449dd0df72d63f1e7c4bd2428e12e4dadb047fa85b4477fc2bf55fd734956" gracePeriod=30
Sep 15 11:52:23.377304 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.377275    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:23.478114 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.478082    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:23.521725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.521647    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb"]
Sep 15 11:52:23.524447 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.524401    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b8fbdd-nrskb"]
Sep 15 11:52:23.565309 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.565223    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:23.597679 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.597655    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"
Sep 15 11:52:23.647329 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.647294    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:23.767958 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.767923    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p77xt\" (UniqueName: \"kubernetes.io/projected/2233a306-f57a-47aa-8c1f-3d3c5ae642d7-kube-api-access-p77xt\") pod \"2233a306-f57a-47aa-8c1f-3d3c5ae642d7\" (UID: \"2233a306-f57a-47aa-8c1f-3d3c5ae642d7\") "
Sep 15 11:52:23.768139 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.767992    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2233a306-f57a-47aa-8c1f-3d3c5ae642d7-plugins-config-volume\") pod \"2233a306-f57a-47aa-8c1f-3d3c5ae642d7\" (UID: \"2233a306-f57a-47aa-8c1f-3d3c5ae642d7\") "
Sep 15 11:52:23.768450 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.768414    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2233a306-f57a-47aa-8c1f-3d3c5ae642d7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2233a306-f57a-47aa-8c1f-3d3c5ae642d7" (UID: "2233a306-f57a-47aa-8c1f-3d3c5ae642d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:52:23.770083 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.770056    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2233a306-f57a-47aa-8c1f-3d3c5ae642d7-kube-api-access-p77xt" (OuterVolumeSpecName: "kube-api-access-p77xt") pod "2233a306-f57a-47aa-8c1f-3d3c5ae642d7" (UID: "2233a306-f57a-47aa-8c1f-3d3c5ae642d7"). InnerVolumeSpecName "kube-api-access-p77xt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:52:23.869766 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.869733    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2233a306-f57a-47aa-8c1f-3d3c5ae642d7-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:23.869766 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.869762    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p77xt\" (UniqueName: \"kubernetes.io/projected/2233a306-f57a-47aa-8c1f-3d3c5ae642d7-kube-api-access-p77xt\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:23.876940 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:23.876913    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:24.269047 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:24.268971    2560 generic.go:356] "Generic (PLEG): container finished" podID="2233a306-f57a-47aa-8c1f-3d3c5ae642d7" containerID="1f3449dd0df72d63f1e7c4bd2428e12e4dadb047fa85b4477fc2bf55fd734956" exitCode=0
Sep 15 11:52:24.269047 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:24.269010    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw" event={"ID":"2233a306-f57a-47aa-8c1f-3d3c5ae642d7","Type":"ContainerDied","Data":"1f3449dd0df72d63f1e7c4bd2428e12e4dadb047fa85b4477fc2bf55fd734956"}
Sep 15 11:52:24.269047 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:24.269027    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"
Sep 15 11:52:24.269344 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:24.269051    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw" event={"ID":"2233a306-f57a-47aa-8c1f-3d3c5ae642d7","Type":"ContainerDied","Data":"82fdd8b44a2dd5921ed5924832c9187f5de94b67f79667fe4ff2ccadf7cad19e"}
Sep 15 11:52:24.269344 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:24.269072    2560 scope.go:117] "RemoveContainer" containerID="1f3449dd0df72d63f1e7c4bd2428e12e4dadb047fa85b4477fc2bf55fd734956"
Sep 15 11:52:24.279543 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:24.279524    2560 scope.go:117] "RemoveContainer" containerID="1f3449dd0df72d63f1e7c4bd2428e12e4dadb047fa85b4477fc2bf55fd734956"
Sep 15 11:52:24.279846 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:52:24.279826    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f3449dd0df72d63f1e7c4bd2428e12e4dadb047fa85b4477fc2bf55fd734956\": container with ID starting with 1f3449dd0df72d63f1e7c4bd2428e12e4dadb047fa85b4477fc2bf55fd734956 not found: ID does not exist" containerID="1f3449dd0df72d63f1e7c4bd2428e12e4dadb047fa85b4477fc2bf55fd734956"
Sep 15 11:52:24.279935 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:24.279858    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f3449dd0df72d63f1e7c4bd2428e12e4dadb047fa85b4477fc2bf55fd734956"} err="failed to get container status \"1f3449dd0df72d63f1e7c4bd2428e12e4dadb047fa85b4477fc2bf55fd734956\": rpc error: code = NotFound desc = could not find container \"1f3449dd0df72d63f1e7c4bd2428e12e4dadb047fa85b4477fc2bf55fd734956\": container with ID starting with 1f3449dd0df72d63f1e7c4bd2428e12e4dadb047fa85b4477fc2bf55fd734956 not found: ID does not exist"
Sep 15 11:52:24.291875 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:24.291850    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"]
Sep 15 11:52:24.298114 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:24.298086    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b8fbdd-69dc8955f-zbjbw"]
Sep 15 11:52:24.305183 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:24.305154    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:24.991204 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:24.991171    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:25.041063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.041022    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e" path="/var/lib/kubelet/pods/0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e/volumes"
Sep 15 11:52:25.041513 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.041496    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2233a306-f57a-47aa-8c1f-3d3c5ae642d7" path="/var/lib/kubelet/pods/2233a306-f57a-47aa-8c1f-3d3c5ae642d7/volumes"
Sep 15 11:52:25.041814 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.041800    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4e14abc-b9da-426e-9f2c-cdfc66c59637" path="/var/lib/kubelet/pods/b4e14abc-b9da-426e-9f2c-cdfc66c59637/volumes"
Sep 15 11:52:25.658710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.658683    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:52:25.783857 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.783778    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:25.789718 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.789688    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-credential-socket\") pod \"152bfca4-485b-4870-92fb-c41e53ed1e3d\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") "
Sep 15 11:52:25.789885 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.789736    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfpwh\" (UniqueName: \"kubernetes.io/projected/152bfca4-485b-4870-92fb-c41e53ed1e3d-kube-api-access-vfpwh\") pod \"152bfca4-485b-4870-92fb-c41e53ed1e3d\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") "
Sep 15 11:52:25.789885 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.789783    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-podinfo\") pod \"152bfca4-485b-4870-92fb-c41e53ed1e3d\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") "
Sep 15 11:52:25.790001 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.789896    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/152bfca4-485b-4870-92fb-c41e53ed1e3d-istiod-ca-cert\") pod \"152bfca4-485b-4870-92fb-c41e53ed1e3d\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") "
Sep 15 11:52:25.790001 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.789934    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-workload-socket\") pod \"152bfca4-485b-4870-92fb-c41e53ed1e3d\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") "
Sep 15 11:52:25.790001 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.789966    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-envoy\") pod \"152bfca4-485b-4870-92fb-c41e53ed1e3d\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") "
Sep 15 11:52:25.790001 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.789993    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-token\") pod \"152bfca4-485b-4870-92fb-c41e53ed1e3d\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") "
Sep 15 11:52:25.790370 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.790014    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-data\") pod \"152bfca4-485b-4870-92fb-c41e53ed1e3d\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") "
Sep 15 11:52:25.790370 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.790100    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-workload-certs\") pod \"152bfca4-485b-4870-92fb-c41e53ed1e3d\" (UID: \"152bfca4-485b-4870-92fb-c41e53ed1e3d\") "
Sep 15 11:52:25.790370 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.790324    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "152bfca4-485b-4870-92fb-c41e53ed1e3d" (UID: "152bfca4-485b-4870-92fb-c41e53ed1e3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:25.791627 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.790542    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:25.791763 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.791728    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "152bfca4-485b-4870-92fb-c41e53ed1e3d" (UID: "152bfca4-485b-4870-92fb-c41e53ed1e3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:25.791977 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.791949    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/152bfca4-485b-4870-92fb-c41e53ed1e3d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "152bfca4-485b-4870-92fb-c41e53ed1e3d" (UID: "152bfca4-485b-4870-92fb-c41e53ed1e3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:52:25.792063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.791982    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-data" (OuterVolumeSpecName: "istio-data") pod "152bfca4-485b-4870-92fb-c41e53ed1e3d" (UID: "152bfca4-485b-4870-92fb-c41e53ed1e3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:25.792186 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.792167    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "152bfca4-485b-4870-92fb-c41e53ed1e3d" (UID: "152bfca4-485b-4870-92fb-c41e53ed1e3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:25.792417 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.792382    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/152bfca4-485b-4870-92fb-c41e53ed1e3d-kube-api-access-vfpwh" (OuterVolumeSpecName: "kube-api-access-vfpwh") pod "152bfca4-485b-4870-92fb-c41e53ed1e3d" (UID: "152bfca4-485b-4870-92fb-c41e53ed1e3d"). InnerVolumeSpecName "kube-api-access-vfpwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:52:25.793599 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.793541    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "152bfca4-485b-4870-92fb-c41e53ed1e3d" (UID: "152bfca4-485b-4870-92fb-c41e53ed1e3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:25.793774 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.793750    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "152bfca4-485b-4870-92fb-c41e53ed1e3d" (UID: "152bfca4-485b-4870-92fb-c41e53ed1e3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:52:25.793877 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.793841    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-token" (OuterVolumeSpecName: "istio-token") pod "152bfca4-485b-4870-92fb-c41e53ed1e3d" (UID: "152bfca4-485b-4870-92fb-c41e53ed1e3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:52:25.891393 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.891348    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/152bfca4-485b-4870-92fb-c41e53ed1e3d-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:25.891393 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.891375    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:25.891393 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.891384    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:25.891393 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.891392    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:25.891393 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.891402    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:25.891841 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.891410    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/152bfca4-485b-4870-92fb-c41e53ed1e3d-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:25.891841 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.891417    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfpwh\" (UniqueName: \"kubernetes.io/projected/152bfca4-485b-4870-92fb-c41e53ed1e3d-kube-api-access-vfpwh\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:25.891841 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.891426    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/152bfca4-485b-4870-92fb-c41e53ed1e3d-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:25.989270 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:25.989232    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:26.283460 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:26.283427    2560 generic.go:356] "Generic (PLEG): container finished" podID="152bfca4-485b-4870-92fb-c41e53ed1e3d" containerID="e1c6707dd3e6ec1e6dede3526f78eedfad18dc0a687ce70d0645905deb625ca4" exitCode=0
Sep 15 11:52:26.283905 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:26.283476    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"
Sep 15 11:52:26.283905 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:26.283515    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw" event={"ID":"152bfca4-485b-4870-92fb-c41e53ed1e3d","Type":"ContainerDied","Data":"e1c6707dd3e6ec1e6dede3526f78eedfad18dc0a687ce70d0645905deb625ca4"}
Sep 15 11:52:26.283905 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:26.283590    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw" event={"ID":"152bfca4-485b-4870-92fb-c41e53ed1e3d","Type":"ContainerDied","Data":"bb8ce7f4ed55a27f1f8007e97fe92c3308726d9b486afc46b409617dd38b8485"}
Sep 15 11:52:26.283905 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:26.283614    2560 scope.go:117] "RemoveContainer" containerID="e1c6707dd3e6ec1e6dede3526f78eedfad18dc0a687ce70d0645905deb625ca4"
Sep 15 11:52:26.293911 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:26.293888    2560 scope.go:117] "RemoveContainer" containerID="4185c467f67c2335e39894010b64c78275ab539d9d2a8baca563c7868e391526"
Sep 15 11:52:26.302770 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:26.302749    2560 scope.go:117] "RemoveContainer" containerID="e1c6707dd3e6ec1e6dede3526f78eedfad18dc0a687ce70d0645905deb625ca4"
Sep 15 11:52:26.303073 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:52:26.303051    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1c6707dd3e6ec1e6dede3526f78eedfad18dc0a687ce70d0645905deb625ca4\": container with ID starting with e1c6707dd3e6ec1e6dede3526f78eedfad18dc0a687ce70d0645905deb625ca4 not found: ID does not exist" containerID="e1c6707dd3e6ec1e6dede3526f78eedfad18dc0a687ce70d0645905deb625ca4"
Sep 15 11:52:26.303135 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:26.303112    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1c6707dd3e6ec1e6dede3526f78eedfad18dc0a687ce70d0645905deb625ca4"} err="failed to get container status \"e1c6707dd3e6ec1e6dede3526f78eedfad18dc0a687ce70d0645905deb625ca4\": rpc error: code = NotFound desc = could not find container \"e1c6707dd3e6ec1e6dede3526f78eedfad18dc0a687ce70d0645905deb625ca4\": container with ID starting with e1c6707dd3e6ec1e6dede3526f78eedfad18dc0a687ce70d0645905deb625ca4 not found: ID does not exist"
Sep 15 11:52:26.303188 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:26.303138    2560 scope.go:117] "RemoveContainer" containerID="4185c467f67c2335e39894010b64c78275ab539d9d2a8baca563c7868e391526"
Sep 15 11:52:26.303420 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:52:26.303393    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4185c467f67c2335e39894010b64c78275ab539d9d2a8baca563c7868e391526\": container with ID starting with 4185c467f67c2335e39894010b64c78275ab539d9d2a8baca563c7868e391526 not found: ID does not exist" containerID="4185c467f67c2335e39894010b64c78275ab539d9d2a8baca563c7868e391526"
Sep 15 11:52:26.303511 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:26.303427    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4185c467f67c2335e39894010b64c78275ab539d9d2a8baca563c7868e391526"} err="failed to get container status \"4185c467f67c2335e39894010b64c78275ab539d9d2a8baca563c7868e391526\": rpc error: code = NotFound desc = could not find container \"4185c467f67c2335e39894010b64c78275ab539d9d2a8baca563c7868e391526\": container with ID starting with 4185c467f67c2335e39894010b64c78275ab539d9d2a8baca563c7868e391526 not found: ID does not exist"
Sep 15 11:52:26.306553 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:26.306531    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"]
Sep 15 11:52:26.312803 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:26.312774    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-2f0e97-openshift-default-9d8c58fd4-6tjzw"]
Sep 15 11:52:26.526395 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:26.526357    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:27.035436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:27.035403    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="152bfca4-485b-4870-92fb-c41e53ed1e3d" path="/var/lib/kubelet/pods/152bfca4-485b-4870-92fb-c41e53ed1e3d/volumes"
Sep 15 11:52:27.392789 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:27.392752    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:29.572900 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:29.572870    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:29.673801 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:29.673760    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:29.905060 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:29.905025    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"]
Sep 15 11:52:29.905282 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:29.905256    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4" podUID="0582ae44-03a9-4a95-841d-ef551a58fc19" containerName="istio-proxy" containerID="cri-o://01f1129c11173137862446600fcecbf7783e3fd28646d0c5fc530afd97af1103" gracePeriod=30
Sep 15 11:52:29.919068 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:29.919043    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:30.872804 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:30.872751    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:31.436722 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:31.436672    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:31.500363 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:31.500322    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:31.566093 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:31.566060    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:31.632201 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:31.632165    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:31.689802 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:31.689702    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:31.987092 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:31.986986    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:32.091416 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:32.091368    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:32.905506 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:32.905460    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:33.667058 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:33.667022    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:33.804793 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:33.804753    2560 patch_prober.go:28] interesting pod/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 11:52:33.804957 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:33.804840    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4" podUID="0582ae44-03a9-4a95-841d-ef551a58fc19" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 11:52:34.668944 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:34.668912    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:35.170411 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.170386    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:52:35.187519 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.187395    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4gjx\" (UniqueName: \"kubernetes.io/projected/0582ae44-03a9-4a95-841d-ef551a58fc19-kube-api-access-r4gjx\") pod \"0582ae44-03a9-4a95-841d-ef551a58fc19\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") "
Sep 15 11:52:35.187519 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.187470    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0582ae44-03a9-4a95-841d-ef551a58fc19-istiod-ca-cert\") pod \"0582ae44-03a9-4a95-841d-ef551a58fc19\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") "
Sep 15 11:52:35.187519 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.187503    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-envoy\") pod \"0582ae44-03a9-4a95-841d-ef551a58fc19\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") "
Sep 15 11:52:35.188416 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.187543    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-workload-certs\") pod \"0582ae44-03a9-4a95-841d-ef551a58fc19\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") "
Sep 15 11:52:35.188416 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.187603    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-workload-socket\") pod \"0582ae44-03a9-4a95-841d-ef551a58fc19\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") "
Sep 15 11:52:35.188416 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.187647    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-token\") pod \"0582ae44-03a9-4a95-841d-ef551a58fc19\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") "
Sep 15 11:52:35.188416 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.187700    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-credential-socket\") pod \"0582ae44-03a9-4a95-841d-ef551a58fc19\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") "
Sep 15 11:52:35.188416 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.187732    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-podinfo\") pod \"0582ae44-03a9-4a95-841d-ef551a58fc19\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") "
Sep 15 11:52:35.188416 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.187885    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-data\") pod \"0582ae44-03a9-4a95-841d-ef551a58fc19\" (UID: \"0582ae44-03a9-4a95-841d-ef551a58fc19\") "
Sep 15 11:52:35.188876 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.188492    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-data" (OuterVolumeSpecName: "istio-data") pod "0582ae44-03a9-4a95-841d-ef551a58fc19" (UID: "0582ae44-03a9-4a95-841d-ef551a58fc19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:35.188876 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.188833    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0582ae44-03a9-4a95-841d-ef551a58fc19" (UID: "0582ae44-03a9-4a95-841d-ef551a58fc19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:35.189073 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.189039    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0582ae44-03a9-4a95-841d-ef551a58fc19" (UID: "0582ae44-03a9-4a95-841d-ef551a58fc19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:35.189631 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.189145    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0582ae44-03a9-4a95-841d-ef551a58fc19-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0582ae44-03a9-4a95-841d-ef551a58fc19" (UID: "0582ae44-03a9-4a95-841d-ef551a58fc19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:52:35.189631 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.189592    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0582ae44-03a9-4a95-841d-ef551a58fc19" (UID: "0582ae44-03a9-4a95-841d-ef551a58fc19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:35.191429 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.191389    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0582ae44-03a9-4a95-841d-ef551a58fc19" (UID: "0582ae44-03a9-4a95-841d-ef551a58fc19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:35.192032 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.191990    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0582ae44-03a9-4a95-841d-ef551a58fc19-kube-api-access-r4gjx" (OuterVolumeSpecName: "kube-api-access-r4gjx") pod "0582ae44-03a9-4a95-841d-ef551a58fc19" (UID: "0582ae44-03a9-4a95-841d-ef551a58fc19"). InnerVolumeSpecName "kube-api-access-r4gjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:52:35.192778 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.192746    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0582ae44-03a9-4a95-841d-ef551a58fc19" (UID: "0582ae44-03a9-4a95-841d-ef551a58fc19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:52:35.196067 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.196029    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-token" (OuterVolumeSpecName: "istio-token") pod "0582ae44-03a9-4a95-841d-ef551a58fc19" (UID: "0582ae44-03a9-4a95-841d-ef551a58fc19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:52:35.289659 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.289619    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0582ae44-03a9-4a95-841d-ef551a58fc19-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:35.289659 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.289657    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:35.289874 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.289671    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:35.289874 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.289683    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:35.289874 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.289695    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:35.289874 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.289707    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:35.289874 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.289719    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:35.289874 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.289731    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0582ae44-03a9-4a95-841d-ef551a58fc19-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:35.289874 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.289743    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4gjx\" (UniqueName: \"kubernetes.io/projected/0582ae44-03a9-4a95-841d-ef551a58fc19-kube-api-access-r4gjx\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:35.341778 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.341746    2560 generic.go:356] "Generic (PLEG): container finished" podID="0582ae44-03a9-4a95-841d-ef551a58fc19" containerID="01f1129c11173137862446600fcecbf7783e3fd28646d0c5fc530afd97af1103" exitCode=0
Sep 15 11:52:35.341926 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.341795    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"
Sep 15 11:52:35.341926 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.341825    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4" event={"ID":"0582ae44-03a9-4a95-841d-ef551a58fc19","Type":"ContainerDied","Data":"01f1129c11173137862446600fcecbf7783e3fd28646d0c5fc530afd97af1103"}
Sep 15 11:52:35.341926 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.341889    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4" event={"ID":"0582ae44-03a9-4a95-841d-ef551a58fc19","Type":"ContainerDied","Data":"d546bcee4e666f6940a6a5e4ce0f155c5e4299624d0f7f2d7e99b0fcd8f5d542"}
Sep 15 11:52:35.341926 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.341919    2560 scope.go:117] "RemoveContainer" containerID="01f1129c11173137862446600fcecbf7783e3fd28646d0c5fc530afd97af1103"
Sep 15 11:52:35.351044 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.351026    2560 scope.go:117] "RemoveContainer" containerID="ae364f9002995116d5d754ab15b26327b1225fe850b5c6ac87ee9cff0207d797"
Sep 15 11:52:35.358960 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.358940    2560 scope.go:117] "RemoveContainer" containerID="01f1129c11173137862446600fcecbf7783e3fd28646d0c5fc530afd97af1103"
Sep 15 11:52:35.359227 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:52:35.359210    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01f1129c11173137862446600fcecbf7783e3fd28646d0c5fc530afd97af1103\": container with ID starting with 01f1129c11173137862446600fcecbf7783e3fd28646d0c5fc530afd97af1103 not found: ID does not exist" containerID="01f1129c11173137862446600fcecbf7783e3fd28646d0c5fc530afd97af1103"
Sep 15 11:52:35.359271 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.359235    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01f1129c11173137862446600fcecbf7783e3fd28646d0c5fc530afd97af1103"} err="failed to get container status \"01f1129c11173137862446600fcecbf7783e3fd28646d0c5fc530afd97af1103\": rpc error: code = NotFound desc = could not find container \"01f1129c11173137862446600fcecbf7783e3fd28646d0c5fc530afd97af1103\": container with ID starting with 01f1129c11173137862446600fcecbf7783e3fd28646d0c5fc530afd97af1103 not found: ID does not exist"
Sep 15 11:52:35.359271 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.359252    2560 scope.go:117] "RemoveContainer" containerID="ae364f9002995116d5d754ab15b26327b1225fe850b5c6ac87ee9cff0207d797"
Sep 15 11:52:35.359497 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:52:35.359480    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae364f9002995116d5d754ab15b26327b1225fe850b5c6ac87ee9cff0207d797\": container with ID starting with ae364f9002995116d5d754ab15b26327b1225fe850b5c6ac87ee9cff0207d797 not found: ID does not exist" containerID="ae364f9002995116d5d754ab15b26327b1225fe850b5c6ac87ee9cff0207d797"
Sep 15 11:52:35.359534 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.359503    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae364f9002995116d5d754ab15b26327b1225fe850b5c6ac87ee9cff0207d797"} err="failed to get container status \"ae364f9002995116d5d754ab15b26327b1225fe850b5c6ac87ee9cff0207d797\": rpc error: code = NotFound desc = could not find container \"ae364f9002995116d5d754ab15b26327b1225fe850b5c6ac87ee9cff0207d797\": container with ID starting with ae364f9002995116d5d754ab15b26327b1225fe850b5c6ac87ee9cff0207d797 not found: ID does not exist"
Sep 15 11:52:35.363553 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.363532    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"]
Sep 15 11:52:35.374534 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:35.374511    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b8fbdd-openshift-default-575d4f777b-g7hx4"]
Sep 15 11:52:37.035507 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:37.035472    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0582ae44-03a9-4a95-841d-ef551a58fc19" path="/var/lib/kubelet/pods/0582ae44-03a9-4a95-841d-ef551a58fc19/volumes"
Sep 15 11:52:37.708484 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:37.708448    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:37.768494 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:37.768453    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:38.055837 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:38.055761    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"]
Sep 15 11:52:38.056223 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:38.056039    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw" podUID="af3d83d7-8709-4154-976f-695994a8cebb" containerName="istio-proxy" containerID="cri-o://0fb28d26848e4a5e5e38eb4e8e22a3fb0fccc2a2bda2c0b0e89869382933f19f" gracePeriod=30
Sep 15 11:52:38.094398 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:38.094360    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:39.175102 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:39.175061    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:39.858118 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:39.858081    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:40.491534 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:40.491487    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:41.279937 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:41.279893    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:43.312873 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.312849    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:52:43.370395 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.370366    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-workload-certs\") pod \"af3d83d7-8709-4154-976f-695994a8cebb\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") "
Sep 15 11:52:43.370580 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.370419    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af3d83d7-8709-4154-976f-695994a8cebb-istio-podinfo\") pod \"af3d83d7-8709-4154-976f-695994a8cebb\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") "
Sep 15 11:52:43.370580 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.370446    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-istio-envoy\") pod \"af3d83d7-8709-4154-976f-695994a8cebb\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") "
Sep 15 11:52:43.370580 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.370496    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af3d83d7-8709-4154-976f-695994a8cebb-istio-token\") pod \"af3d83d7-8709-4154-976f-695994a8cebb\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") "
Sep 15 11:52:43.370580 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.370519    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdn2s\" (UniqueName: \"kubernetes.io/projected/af3d83d7-8709-4154-976f-695994a8cebb-kube-api-access-qdn2s\") pod \"af3d83d7-8709-4154-976f-695994a8cebb\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") "
Sep 15 11:52:43.370580 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.370547    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-istio-data\") pod \"af3d83d7-8709-4154-976f-695994a8cebb\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") "
Sep 15 11:52:43.370844 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.370652    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-workload-socket\") pod \"af3d83d7-8709-4154-976f-695994a8cebb\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") "
Sep 15 11:52:43.370844 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.370659    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "af3d83d7-8709-4154-976f-695994a8cebb" (UID: "af3d83d7-8709-4154-976f-695994a8cebb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:43.370844 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.370682    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-credential-socket\") pod \"af3d83d7-8709-4154-976f-695994a8cebb\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") "
Sep 15 11:52:43.370844 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.370713    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af3d83d7-8709-4154-976f-695994a8cebb-istiod-ca-cert\") pod \"af3d83d7-8709-4154-976f-695994a8cebb\" (UID: \"af3d83d7-8709-4154-976f-695994a8cebb\") "
Sep 15 11:52:43.371063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.370907    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "af3d83d7-8709-4154-976f-695994a8cebb" (UID: "af3d83d7-8709-4154-976f-695994a8cebb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:43.371063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.370928    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-istio-data" (OuterVolumeSpecName: "istio-data") pod "af3d83d7-8709-4154-976f-695994a8cebb" (UID: "af3d83d7-8709-4154-976f-695994a8cebb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:43.371169 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.371083    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "af3d83d7-8709-4154-976f-695994a8cebb" (UID: "af3d83d7-8709-4154-976f-695994a8cebb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:43.371228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.371216    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:43.371281 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.371235    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:43.371281 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.371249    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:43.371281 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.371264    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:43.371583 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.371537    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af3d83d7-8709-4154-976f-695994a8cebb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "af3d83d7-8709-4154-976f-695994a8cebb" (UID: "af3d83d7-8709-4154-976f-695994a8cebb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:52:43.372765 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.372736    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/af3d83d7-8709-4154-976f-695994a8cebb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "af3d83d7-8709-4154-976f-695994a8cebb" (UID: "af3d83d7-8709-4154-976f-695994a8cebb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:52:43.372876 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.372753    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af3d83d7-8709-4154-976f-695994a8cebb-kube-api-access-qdn2s" (OuterVolumeSpecName: "kube-api-access-qdn2s") pod "af3d83d7-8709-4154-976f-695994a8cebb" (UID: "af3d83d7-8709-4154-976f-695994a8cebb"). InnerVolumeSpecName "kube-api-access-qdn2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:52:43.372876 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.372772    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "af3d83d7-8709-4154-976f-695994a8cebb" (UID: "af3d83d7-8709-4154-976f-695994a8cebb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:52:43.372876 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.372827    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af3d83d7-8709-4154-976f-695994a8cebb-istio-token" (OuterVolumeSpecName: "istio-token") pod "af3d83d7-8709-4154-976f-695994a8cebb" (UID: "af3d83d7-8709-4154-976f-695994a8cebb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:52:43.394629 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.394599    2560 generic.go:356] "Generic (PLEG): container finished" podID="af3d83d7-8709-4154-976f-695994a8cebb" containerID="0fb28d26848e4a5e5e38eb4e8e22a3fb0fccc2a2bda2c0b0e89869382933f19f" exitCode=0
Sep 15 11:52:43.394738 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.394676    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"
Sep 15 11:52:43.394738 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.394684    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw" event={"ID":"af3d83d7-8709-4154-976f-695994a8cebb","Type":"ContainerDied","Data":"0fb28d26848e4a5e5e38eb4e8e22a3fb0fccc2a2bda2c0b0e89869382933f19f"}
Sep 15 11:52:43.394859 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.394745    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw" event={"ID":"af3d83d7-8709-4154-976f-695994a8cebb","Type":"ContainerDied","Data":"1d1f2b312b28215db2fa7b0ba64a50a19f655ec0744cc9c1781169edd730b686"}
Sep 15 11:52:43.394859 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.394776    2560 scope.go:117] "RemoveContainer" containerID="0fb28d26848e4a5e5e38eb4e8e22a3fb0fccc2a2bda2c0b0e89869382933f19f"
Sep 15 11:52:43.404592 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.404573    2560 scope.go:117] "RemoveContainer" containerID="af80e6eb002febdb0a92e37fc026fb323a7ee73ac331095d43a85b06c641ed12"
Sep 15 11:52:43.412823 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.412806    2560 scope.go:117] "RemoveContainer" containerID="0fb28d26848e4a5e5e38eb4e8e22a3fb0fccc2a2bda2c0b0e89869382933f19f"
Sep 15 11:52:43.413065 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:52:43.413044    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fb28d26848e4a5e5e38eb4e8e22a3fb0fccc2a2bda2c0b0e89869382933f19f\": container with ID starting with 0fb28d26848e4a5e5e38eb4e8e22a3fb0fccc2a2bda2c0b0e89869382933f19f not found: ID does not exist" containerID="0fb28d26848e4a5e5e38eb4e8e22a3fb0fccc2a2bda2c0b0e89869382933f19f"
Sep 15 11:52:43.413119 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.413074    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fb28d26848e4a5e5e38eb4e8e22a3fb0fccc2a2bda2c0b0e89869382933f19f"} err="failed to get container status \"0fb28d26848e4a5e5e38eb4e8e22a3fb0fccc2a2bda2c0b0e89869382933f19f\": rpc error: code = NotFound desc = could not find container \"0fb28d26848e4a5e5e38eb4e8e22a3fb0fccc2a2bda2c0b0e89869382933f19f\": container with ID starting with 0fb28d26848e4a5e5e38eb4e8e22a3fb0fccc2a2bda2c0b0e89869382933f19f not found: ID does not exist"
Sep 15 11:52:43.413119 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.413090    2560 scope.go:117] "RemoveContainer" containerID="af80e6eb002febdb0a92e37fc026fb323a7ee73ac331095d43a85b06c641ed12"
Sep 15 11:52:43.413376 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:52:43.413352    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af80e6eb002febdb0a92e37fc026fb323a7ee73ac331095d43a85b06c641ed12\": container with ID starting with af80e6eb002febdb0a92e37fc026fb323a7ee73ac331095d43a85b06c641ed12 not found: ID does not exist" containerID="af80e6eb002febdb0a92e37fc026fb323a7ee73ac331095d43a85b06c641ed12"
Sep 15 11:52:43.413476 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.413381    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af80e6eb002febdb0a92e37fc026fb323a7ee73ac331095d43a85b06c641ed12"} err="failed to get container status \"af80e6eb002febdb0a92e37fc026fb323a7ee73ac331095d43a85b06c641ed12\": rpc error: code = NotFound desc = could not find container \"af80e6eb002febdb0a92e37fc026fb323a7ee73ac331095d43a85b06c641ed12\": container with ID starting with af80e6eb002febdb0a92e37fc026fb323a7ee73ac331095d43a85b06c641ed12 not found: ID does not exist"
Sep 15 11:52:43.416615 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.416594    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"]
Sep 15 11:52:43.420105 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.420084    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5a8d6f-openshift-default-57c8dcc7c8-9mzqw"]
Sep 15 11:52:43.471780 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.471757    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af3d83d7-8709-4154-976f-695994a8cebb-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:43.471780 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.471780    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdn2s\" (UniqueName: \"kubernetes.io/projected/af3d83d7-8709-4154-976f-695994a8cebb-kube-api-access-qdn2s\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:43.471913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.471791    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af3d83d7-8709-4154-976f-695994a8cebb-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:43.471913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.471801    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af3d83d7-8709-4154-976f-695994a8cebb-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:43.471913 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:43.471809    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af3d83d7-8709-4154-976f-695994a8cebb-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:45.035143 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:45.035108    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af3d83d7-8709-4154-976f-695994a8cebb" path="/var/lib/kubelet/pods/af3d83d7-8709-4154-976f-695994a8cebb/volumes"
Sep 15 11:52:48.807093 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:48.807053    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:48.916581 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:48.916537    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:48.958323 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:48.958293    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d63075-29824545-lwlw8"]
Sep 15 11:52:48.962688 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:48.962662    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d63075-29824545-lwlw8"]
Sep 15 11:52:48.997007 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:48.996974    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"]
Sep 15 11:52:48.997259 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:48.997227    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr" podUID="24bb9262-01ea-495e-a22c-5c60c0ea27d5" containerName="payload-pre-processing" containerID="cri-o://d67a8d724ccb28a9f8ad971494877fcc65509d26c7d99d89eb2402a28e4e15f0" gracePeriod=30
Sep 15 11:52:49.000428 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.000401    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:49.037355 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.037327    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c106311e-7d48-411e-8eb2-7e539449d9a3" path="/var/lib/kubelet/pods/c106311e-7d48-411e-8eb2-7e539449d9a3/volumes"
Sep 15 11:52:49.070543 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.070467    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:49.169448 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.169376    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:49.231954 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.231922    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:49.252182 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.252163    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"
Sep 15 11:52:49.331658 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.330015    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24bb9262-01ea-495e-a22c-5c60c0ea27d5-plugins-config-volume\") pod \"24bb9262-01ea-495e-a22c-5c60c0ea27d5\" (UID: \"24bb9262-01ea-495e-a22c-5c60c0ea27d5\") "
Sep 15 11:52:49.331658 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.330134    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsxl7\" (UniqueName: \"kubernetes.io/projected/24bb9262-01ea-495e-a22c-5c60c0ea27d5-kube-api-access-bsxl7\") pod \"24bb9262-01ea-495e-a22c-5c60c0ea27d5\" (UID: \"24bb9262-01ea-495e-a22c-5c60c0ea27d5\") "
Sep 15 11:52:49.331658 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.331127    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24bb9262-01ea-495e-a22c-5c60c0ea27d5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "24bb9262-01ea-495e-a22c-5c60c0ea27d5" (UID: "24bb9262-01ea-495e-a22c-5c60c0ea27d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:52:49.333905 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.333874    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24bb9262-01ea-495e-a22c-5c60c0ea27d5-kube-api-access-bsxl7" (OuterVolumeSpecName: "kube-api-access-bsxl7") pod "24bb9262-01ea-495e-a22c-5c60c0ea27d5" (UID: "24bb9262-01ea-495e-a22c-5c60c0ea27d5"). InnerVolumeSpecName "kube-api-access-bsxl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:52:49.431359 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.431327    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsxl7\" (UniqueName: \"kubernetes.io/projected/24bb9262-01ea-495e-a22c-5c60c0ea27d5-kube-api-access-bsxl7\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:49.431359 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.431354    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24bb9262-01ea-495e-a22c-5c60c0ea27d5-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:52:49.432613 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.432587    2560 generic.go:356] "Generic (PLEG): container finished" podID="24bb9262-01ea-495e-a22c-5c60c0ea27d5" containerID="d67a8d724ccb28a9f8ad971494877fcc65509d26c7d99d89eb2402a28e4e15f0" exitCode=0
Sep 15 11:52:49.432743 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.432686    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"
Sep 15 11:52:49.432797 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.432737    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr" event={"ID":"24bb9262-01ea-495e-a22c-5c60c0ea27d5","Type":"ContainerDied","Data":"d67a8d724ccb28a9f8ad971494877fcc65509d26c7d99d89eb2402a28e4e15f0"}
Sep 15 11:52:49.432843 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.432800    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr" event={"ID":"24bb9262-01ea-495e-a22c-5c60c0ea27d5","Type":"ContainerDied","Data":"305697e4e56b4db3070fe0f7462b8d0b3b113e04e4682965678e1d5e8fcf4723"}
Sep 15 11:52:49.432843 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.432833    2560 scope.go:117] "RemoveContainer" containerID="d67a8d724ccb28a9f8ad971494877fcc65509d26c7d99d89eb2402a28e4e15f0"
Sep 15 11:52:49.446254 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.446233    2560 scope.go:117] "RemoveContainer" containerID="d67a8d724ccb28a9f8ad971494877fcc65509d26c7d99d89eb2402a28e4e15f0"
Sep 15 11:52:49.446589 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:52:49.446551    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d67a8d724ccb28a9f8ad971494877fcc65509d26c7d99d89eb2402a28e4e15f0\": container with ID starting with d67a8d724ccb28a9f8ad971494877fcc65509d26c7d99d89eb2402a28e4e15f0 not found: ID does not exist" containerID="d67a8d724ccb28a9f8ad971494877fcc65509d26c7d99d89eb2402a28e4e15f0"
Sep 15 11:52:49.446689 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.446593    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d67a8d724ccb28a9f8ad971494877fcc65509d26c7d99d89eb2402a28e4e15f0"} err="failed to get container status \"d67a8d724ccb28a9f8ad971494877fcc65509d26c7d99d89eb2402a28e4e15f0\": rpc error: code = NotFound desc = could not find container \"d67a8d724ccb28a9f8ad971494877fcc65509d26c7d99d89eb2402a28e4e15f0\": container with ID starting with d67a8d724ccb28a9f8ad971494877fcc65509d26c7d99d89eb2402a28e4e15f0 not found: ID does not exist"
Sep 15 11:52:49.460402 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.460345    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"]
Sep 15 11:52:49.465625 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.465595    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d63075-59df885f58-8pbzr"]
Sep 15 11:52:49.680965 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.680929    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:49.775276 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:49.775243    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:50.519175 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:50.519129    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:50.619090 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:50.619048    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:51.035661 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:51.035621    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24bb9262-01ea-495e-a22c-5c60c0ea27d5" path="/var/lib/kubelet/pods/24bb9262-01ea-495e-a22c-5c60c0ea27d5/volumes"
Sep 15 11:52:51.308024 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:51.307947    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:55.755109 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:55.755075    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:55.815754 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:55.815720    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:55.999868 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:55.999835    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"]
Sep 15 11:52:56.000107 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:56.000080    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerName="istio-proxy" containerID="cri-o://454c2f4fc36b0d0ed83ca04c3315bb8840603409f80fe51fbb02909e980ad6b1" gracePeriod=30
Sep 15 11:52:56.016659 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:56.016592    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:52:59.739228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:59.739190    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 11:52:59.739623 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:59.739253    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 11:52:59.794489 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:59.794461    2560 scope.go:117] "RemoveContainer" containerID="5152b7c5a96a3304bd1749f59af40fa4091ab226677b6938836e17365cb0210c"
Sep 15 11:52:59.803102 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:59.803083    2560 scope.go:117] "RemoveContainer" containerID="8050f257031b2114f28d3332fa0ccd6fe3ecbe6b0b1b5bae5f2fc34ef1cf61d5"
Sep 15 11:52:59.810903 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:52:59.810884    2560 scope.go:117] "RemoveContainer" containerID="177f13feb56cd9f8b23164dc931cb399f1ca27e5f98d1bc0443f6580f39a68da"
Sep 15 11:53:01.257401 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.257376    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:53:01.348152 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.348128    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-workload-socket\") pod \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") "
Sep 15 11:53:01.348286 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.348159    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istiod-ca-cert\") pod \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") "
Sep 15 11:53:01.348286 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.348197    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t554r\" (UniqueName: \"kubernetes.io/projected/371c85ad-9ab7-4fad-b7be-f13b7d859a43-kube-api-access-t554r\") pod \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") "
Sep 15 11:53:01.348286 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.348227    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-workload-certs\") pod \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") "
Sep 15 11:53:01.348286 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.348255    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-credential-socket\") pod \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") "
Sep 15 11:53:01.348286 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.348282    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-data\") pod \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") "
Sep 15 11:53:01.348547 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.348324    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-envoy\") pod \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") "
Sep 15 11:53:01.348547 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.348393    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "371c85ad-9ab7-4fad-b7be-f13b7d859a43" (UID: "371c85ad-9ab7-4fad-b7be-f13b7d859a43"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:53:01.348547 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.348541    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-token\") pod \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") "
Sep 15 11:53:01.348733 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.348595    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "371c85ad-9ab7-4fad-b7be-f13b7d859a43" (UID: "371c85ad-9ab7-4fad-b7be-f13b7d859a43"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:53:01.348733 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.348629    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-data" (OuterVolumeSpecName: "istio-data") pod "371c85ad-9ab7-4fad-b7be-f13b7d859a43" (UID: "371c85ad-9ab7-4fad-b7be-f13b7d859a43"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:53:01.348733 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.348640    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "371c85ad-9ab7-4fad-b7be-f13b7d859a43" (UID: "371c85ad-9ab7-4fad-b7be-f13b7d859a43"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:53:01.348733 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.348650    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "371c85ad-9ab7-4fad-b7be-f13b7d859a43" (UID: "371c85ad-9ab7-4fad-b7be-f13b7d859a43"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:53:01.348954 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.348838    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-podinfo\") pod \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\" (UID: \"371c85ad-9ab7-4fad-b7be-f13b7d859a43\") "
Sep 15 11:53:01.350637 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.349735    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:53:01.350637 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.349765    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:53:01.350637 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.349779    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:53:01.350637 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.349800    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:53:01.350637 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.349816    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:53:01.350988 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.350958    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-token" (OuterVolumeSpecName: "istio-token") pod "371c85ad-9ab7-4fad-b7be-f13b7d859a43" (UID: "371c85ad-9ab7-4fad-b7be-f13b7d859a43"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:53:01.351125 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.351070    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "371c85ad-9ab7-4fad-b7be-f13b7d859a43" (UID: "371c85ad-9ab7-4fad-b7be-f13b7d859a43"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:53:01.352253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.352229    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/371c85ad-9ab7-4fad-b7be-f13b7d859a43-kube-api-access-t554r" (OuterVolumeSpecName: "kube-api-access-t554r") pod "371c85ad-9ab7-4fad-b7be-f13b7d859a43" (UID: "371c85ad-9ab7-4fad-b7be-f13b7d859a43"). InnerVolumeSpecName "kube-api-access-t554r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:53:01.358261 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.358235    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "371c85ad-9ab7-4fad-b7be-f13b7d859a43" (UID: "371c85ad-9ab7-4fad-b7be-f13b7d859a43"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:53:01.450695 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.450667    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t554r\" (UniqueName: \"kubernetes.io/projected/371c85ad-9ab7-4fad-b7be-f13b7d859a43-kube-api-access-t554r\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:53:01.450695 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.450691    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:53:01.450695 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.450700    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:53:01.450904 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.450709    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/371c85ad-9ab7-4fad-b7be-f13b7d859a43-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:53:01.513658 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.513624    2560 generic.go:356] "Generic (PLEG): container finished" podID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerID="454c2f4fc36b0d0ed83ca04c3315bb8840603409f80fe51fbb02909e980ad6b1" exitCode=0
Sep 15 11:53:01.513816 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.513691    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"
Sep 15 11:53:01.513816 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.513722    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" event={"ID":"371c85ad-9ab7-4fad-b7be-f13b7d859a43","Type":"ContainerDied","Data":"454c2f4fc36b0d0ed83ca04c3315bb8840603409f80fe51fbb02909e980ad6b1"}
Sep 15 11:53:01.513816 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.513805    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh" event={"ID":"371c85ad-9ab7-4fad-b7be-f13b7d859a43","Type":"ContainerDied","Data":"31f28fc697319364ea063f0f0d2cf1347d552a2cfd509270846bcf656dad97ab"}
Sep 15 11:53:01.513998 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.513837    2560 scope.go:117] "RemoveContainer" containerID="454c2f4fc36b0d0ed83ca04c3315bb8840603409f80fe51fbb02909e980ad6b1"
Sep 15 11:53:01.523403 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.523382    2560 scope.go:117] "RemoveContainer" containerID="6d303c4f4806f992d3a5a09af6480ec0c887a328b661dbf4a42d3a08eff8a431"
Sep 15 11:53:01.531103 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.531085    2560 scope.go:117] "RemoveContainer" containerID="454c2f4fc36b0d0ed83ca04c3315bb8840603409f80fe51fbb02909e980ad6b1"
Sep 15 11:53:01.531331 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:53:01.531312    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"454c2f4fc36b0d0ed83ca04c3315bb8840603409f80fe51fbb02909e980ad6b1\": container with ID starting with 454c2f4fc36b0d0ed83ca04c3315bb8840603409f80fe51fbb02909e980ad6b1 not found: ID does not exist" containerID="454c2f4fc36b0d0ed83ca04c3315bb8840603409f80fe51fbb02909e980ad6b1"
Sep 15 11:53:01.531394 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.531339    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454c2f4fc36b0d0ed83ca04c3315bb8840603409f80fe51fbb02909e980ad6b1"} err="failed to get container status \"454c2f4fc36b0d0ed83ca04c3315bb8840603409f80fe51fbb02909e980ad6b1\": rpc error: code = NotFound desc = could not find container \"454c2f4fc36b0d0ed83ca04c3315bb8840603409f80fe51fbb02909e980ad6b1\": container with ID starting with 454c2f4fc36b0d0ed83ca04c3315bb8840603409f80fe51fbb02909e980ad6b1 not found: ID does not exist"
Sep 15 11:53:01.531394 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.531354    2560 scope.go:117] "RemoveContainer" containerID="6d303c4f4806f992d3a5a09af6480ec0c887a328b661dbf4a42d3a08eff8a431"
Sep 15 11:53:01.531601 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:53:01.531585    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d303c4f4806f992d3a5a09af6480ec0c887a328b661dbf4a42d3a08eff8a431\": container with ID starting with 6d303c4f4806f992d3a5a09af6480ec0c887a328b661dbf4a42d3a08eff8a431 not found: ID does not exist" containerID="6d303c4f4806f992d3a5a09af6480ec0c887a328b661dbf4a42d3a08eff8a431"
Sep 15 11:53:01.531649 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.531603    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d303c4f4806f992d3a5a09af6480ec0c887a328b661dbf4a42d3a08eff8a431"} err="failed to get container status \"6d303c4f4806f992d3a5a09af6480ec0c887a328b661dbf4a42d3a08eff8a431\": rpc error: code = NotFound desc = could not find container \"6d303c4f4806f992d3a5a09af6480ec0c887a328b661dbf4a42d3a08eff8a431\": container with ID starting with 6d303c4f4806f992d3a5a09af6480ec0c887a328b661dbf4a42d3a08eff8a431 not found: ID does not exist"
Sep 15 11:53:01.535395 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.535372    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"]
Sep 15 11:53:01.538763 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:01.538743    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d63075-openshift-default-696965bc98-s8gjh"]
Sep 15 11:53:03.034835 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:03.034800    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" path="/var/lib/kubelet/pods/371c85ad-9ab7-4fad-b7be-f13b7d859a43/volumes"
Sep 15 11:53:08.329794 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:08.329752    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:53:09.544923 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:09.544885    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:53:13.953435 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:13.953397    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:53:33.172908 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:33.172822    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:53:51.114973 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:51.114932    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:53:58.844967 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:53:58.844930    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:54:04.296086 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:54:04.296020    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:54:04.396986 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:54:04.396938    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:54:31.704407 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:54:31.704368    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:54:32.784353 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:54:32.784321    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:54:38.186073 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:54:38.186033    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:54:46.245619 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:54:46.245585    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:54:59.678175 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:54:59.678149    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:54:59.682239 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:54:59.682214    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:54:59.871863 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:54:59.871835    2560 scope.go:117] "RemoveContainer" containerID="1c826ba8d9f7b159c1aca96f31b587d4a11c2ba84f2cce4f46fee20d2ac5f66e"
Sep 15 11:55:09.045129 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:09.045096    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:55:39.605248 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:39.605208    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:55:40.972811 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:40.972777    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:55:41.073981 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.073942    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:55:41.344997 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.344964    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"]
Sep 15 11:55:41.345488 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345466    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerName="istio-proxy"
Sep 15 11:55:41.345612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345490    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerName="istio-proxy"
Sep 15 11:55:41.345612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345508    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af3d83d7-8709-4154-976f-695994a8cebb" containerName="istio-proxy"
Sep 15 11:55:41.345612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345515    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="af3d83d7-8709-4154-976f-695994a8cebb" containerName="istio-proxy"
Sep 15 11:55:41.345612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345527    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0582ae44-03a9-4a95-841d-ef551a58fc19" containerName="istio-proxy"
Sep 15 11:55:41.345612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345536    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0582ae44-03a9-4a95-841d-ef551a58fc19" containerName="istio-proxy"
Sep 15 11:55:41.345612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345547    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="152bfca4-485b-4870-92fb-c41e53ed1e3d" containerName="istio-proxy"
Sep 15 11:55:41.345612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345555    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="152bfca4-485b-4870-92fb-c41e53ed1e3d" containerName="istio-proxy"
Sep 15 11:55:41.345612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345580    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2233a306-f57a-47aa-8c1f-3d3c5ae642d7" containerName="payload-processing"
Sep 15 11:55:41.345612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345587    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2233a306-f57a-47aa-8c1f-3d3c5ae642d7" containerName="payload-processing"
Sep 15 11:55:41.345612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345598    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24bb9262-01ea-495e-a22c-5c60c0ea27d5" containerName="payload-pre-processing"
Sep 15 11:55:41.345612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345606    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="24bb9262-01ea-495e-a22c-5c60c0ea27d5" containerName="payload-pre-processing"
Sep 15 11:55:41.345612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345617    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" containerName="istio-proxy"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345625    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="49fbd7b3-8b22-415d-8f2f-243ea054e36f" containerName="istio-proxy"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345638    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af3d83d7-8709-4154-976f-695994a8cebb" containerName="istio-proxy"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345646    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="af3d83d7-8709-4154-976f-695994a8cebb" containerName="istio-proxy"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345662    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f737c66-5417-491d-8c79-9a4649e2eb7a" containerName="istio-proxy"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345670    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f737c66-5417-491d-8c79-9a4649e2eb7a" containerName="istio-proxy"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345697    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e" containerName="revoke-keys"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345706    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e" containerName="revoke-keys"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345805    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="af3d83d7-8709-4154-976f-695994a8cebb" containerName="istio-proxy"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345818    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="152bfca4-485b-4870-92fb-c41e53ed1e3d" containerName="istio-proxy"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345828    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e9f566e-e9e5-4b85-ab54-d9c2a05ecb9e" containerName="revoke-keys"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345838    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerName="istio-proxy"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345855    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0582ae44-03a9-4a95-841d-ef551a58fc19" containerName="istio-proxy"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345862    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="af3d83d7-8709-4154-976f-695994a8cebb" containerName="istio-proxy"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345873    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f737c66-5417-491d-8c79-9a4649e2eb7a" containerName="istio-proxy"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345881    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="24bb9262-01ea-495e-a22c-5c60c0ea27d5" containerName="payload-pre-processing"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345889    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0582ae44-03a9-4a95-841d-ef551a58fc19" containerName="istio-proxy"
Sep 15 11:55:41.346206 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.345899    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2233a306-f57a-47aa-8c1f-3d3c5ae642d7" containerName="payload-processing"
Sep 15 11:55:41.349472 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.349449    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.351882 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.351863    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-173324-openshift-default-dockercfg-j69bs\""
Sep 15 11:55:41.364158 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.364135    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"]
Sep 15 11:55:41.389291 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.389264    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:55:41.394340 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.394311    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/071de3b0-e4d5-43a0-b167-1356abd89f1b-istiod-ca-cert\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.394457 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.394366    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg9bf\" (UniqueName: \"kubernetes.io/projected/071de3b0-e4d5-43a0-b167-1356abd89f1b-kube-api-access-xg9bf\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.394530 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.394472    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-credential-socket\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.394530 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.394501    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-podinfo\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.394530 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.394525    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-workload-socket\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.394669 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.394598    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-envoy\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.394669 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.394616    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-token\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.394669 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.394662    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-data\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.394798 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.394703    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-workload-certs\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.496236 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.496193    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-data\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.496236 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.496231    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-workload-certs\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.496498 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.496281    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/071de3b0-e4d5-43a0-b167-1356abd89f1b-istiod-ca-cert\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.496498 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.496307    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg9bf\" (UniqueName: \"kubernetes.io/projected/071de3b0-e4d5-43a0-b167-1356abd89f1b-kube-api-access-xg9bf\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.496498 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.496339    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-credential-socket\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.496498 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.496363    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-podinfo\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.496498 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.496387    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-workload-socket\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.496498 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.496420    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-envoy\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.496498 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.496445    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-token\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.496834 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.496653    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-data\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.496834 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.496695    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-workload-certs\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.496951 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.496933    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-workload-socket\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.497011 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.496991    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-credential-socket\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.497105 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.497086    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/071de3b0-e4d5-43a0-b167-1356abd89f1b-istiod-ca-cert\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.499277 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.499256    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-envoy\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.499526 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.499509    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-podinfo\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.510166 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.510143    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg9bf\" (UniqueName: \"kubernetes.io/projected/071de3b0-e4d5-43a0-b167-1356abd89f1b-kube-api-access-xg9bf\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.510294 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.510218    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-token\") pod \"e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") " pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.554378 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.554352    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:55:41.659888 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.659807    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:41.944071 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.944042    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"]
Sep 15 11:55:41.946284 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:55:41.946258    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod071de3b0_e4d5_43a0_b167_1356abd89f1b.slice/crio-f01425b171c197da004e6cca939cda7595e1694ed62d440dbcc47ae8b9f7b64f WatchSource:0}: Error finding container f01425b171c197da004e6cca939cda7595e1694ed62d440dbcc47ae8b9f7b64f: Status 404 returned error can't find the container with id f01425b171c197da004e6cca939cda7595e1694ed62d440dbcc47ae8b9f7b64f
Sep 15 11:55:41.948421 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.948386    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:55:41.948519 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.948468    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:55:41.948601 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:41.948515    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:55:42.384611 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:42.384577    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j" event={"ID":"071de3b0-e4d5-43a0-b167-1356abd89f1b","Type":"ContainerStarted","Data":"953301990e5c5e4cfab9aae26b197f19d83cff51aa46b4cef662dd35fedb0aaa"}
Sep 15 11:55:42.385005 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:42.384626    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j" event={"ID":"071de3b0-e4d5-43a0-b167-1356abd89f1b","Type":"ContainerStarted","Data":"f01425b171c197da004e6cca939cda7595e1694ed62d440dbcc47ae8b9f7b64f"}
Sep 15 11:55:42.409436 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:42.409387    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j" podStartSLOduration=1.409371184 podStartE2EDuration="1.409371184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:55:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:55:42.407411666 +0000 UTC m=+1843.900188826" watchObservedRunningTime="2026-09-15 11:55:42.409371184 +0000 UTC m=+1843.902148331"
Sep 15 11:55:42.660853 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:42.660761    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:43.664903 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:43.664867    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:43.711601 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:43.711546    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:55:44.397278 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:44.397242    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:44.398408 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:44.398373    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:55:44.444615 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:44.444557    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:55:44.656550 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:44.656464    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:55:44.813599 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:44.813546    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:55:45.889290 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:45.889251    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:55:46.910646 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:46.910608    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:55:46.988789 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:46.988758    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:55:47.144155 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:47.144114    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:55:47.463825 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:47.463783    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:55:47.570530 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:55:47.570486    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:00.438725 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:00.438672    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 15 11:56:00.439228 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:00.438765    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j" podUID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 15 11:56:02.496002 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:02.495970    2560 generic.go:356] "Generic (PLEG): container finished" podID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerID="953301990e5c5e4cfab9aae26b197f19d83cff51aa46b4cef662dd35fedb0aaa" exitCode=0
Sep 15 11:56:02.496429 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:02.496028    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j" event={"ID":"071de3b0-e4d5-43a0-b167-1356abd89f1b","Type":"ContainerDied","Data":"953301990e5c5e4cfab9aae26b197f19d83cff51aa46b4cef662dd35fedb0aaa"}
Sep 15 11:56:02.496429 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:02.496396    2560 scope.go:117] "RemoveContainer" containerID="953301990e5c5e4cfab9aae26b197f19d83cff51aa46b4cef662dd35fedb0aaa"
Sep 15 11:56:02.497502 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:02.497468    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:56:02.497556 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:02.497542    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:56:02.497616 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:02.497592    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:56:03.511314 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:03.511268    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j" event={"ID":"071de3b0-e4d5-43a0-b167-1356abd89f1b","Type":"ContainerStarted","Data":"411c8e8cbb3027e3ca86963d607a41506ee9e0c6d7a9c5e9c7891f5f3d86692a"}
Sep 15 11:56:03.660629 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:03.660591    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:56:03.660824 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:03.660797    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 15 11:56:03.660910 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:03.660846    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j" podUID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 15 11:56:04.661141 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:04.661105    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 15 11:56:04.661529 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:04.661169    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j" podUID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 15 11:56:05.660451 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:05.660415    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 15 11:56:05.660669 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:05.660481    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j" podUID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 15 11:56:06.660776 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:06.660737    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 15 11:56:06.661168 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:06.660795    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j" podUID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 15 11:56:07.660459 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:07.660425    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 15 11:56:07.660655 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:07.660490    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j" podUID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 15 11:56:08.664197 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:08.664167    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:56:08.664640 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:08.664600    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:56:08.665146 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:08.665120    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:56:09.532420 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:09.532381    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:09.634496 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:09.634466    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:10.204693 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.204654    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:10.355761 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.355727    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:10.507599 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.507490    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"]
Sep 15 11:56:10.508244 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.508218    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerName="istio-proxy"
Sep 15 11:56:10.508244 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.508242    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerName="istio-proxy"
Sep 15 11:56:10.508493 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.508263    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0582ae44-03a9-4a95-841d-ef551a58fc19" containerName="istio-proxy"
Sep 15 11:56:10.508493 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.508271    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0582ae44-03a9-4a95-841d-ef551a58fc19" containerName="istio-proxy"
Sep 15 11:56:10.508493 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.508284    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="152bfca4-485b-4870-92fb-c41e53ed1e3d" containerName="istio-proxy"
Sep 15 11:56:10.508493 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.508293    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="152bfca4-485b-4870-92fb-c41e53ed1e3d" containerName="istio-proxy"
Sep 15 11:56:10.508493 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.508411    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="152bfca4-485b-4870-92fb-c41e53ed1e3d" containerName="istio-proxy"
Sep 15 11:56:10.508493 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.508424    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="371c85ad-9ab7-4fad-b7be-f13b7d859a43" containerName="istio-proxy"
Sep 15 11:56:10.516470 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.516422    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.521066 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.521008    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-d40293-openshift-default-dockercfg-ndb8c\""
Sep 15 11:56:10.524162 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.524135    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"]
Sep 15 11:56:10.547489 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.547451    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:10.581531 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.581507    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-envoy\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.581662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.581539    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-workload-socket\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.581662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.581582    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-credential-socket\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.581662 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.581651    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-podinfo\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.581812 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.581675    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-data\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.581812 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.581703    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-workload-certs\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.581812 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.581801    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-token\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.581986 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.581882    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr2b5\" (UniqueName: \"kubernetes.io/projected/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-kube-api-access-qr2b5\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.581986 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.581935    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istiod-ca-cert\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.683063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.683027    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istiod-ca-cert\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.683266 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.683071    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-envoy\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.683266 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.683091    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-workload-socket\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.683266 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.683105    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-credential-socket\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.683266 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.683154    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-podinfo\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.683266 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.683179    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-data\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.683266 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.683218    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-workload-certs\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.683644 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.683280    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-token\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.683644 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.683318    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr2b5\" (UniqueName: \"kubernetes.io/projected/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-kube-api-access-qr2b5\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.683644 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.683542    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-workload-socket\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.683644 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.683620    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-credential-socket\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.683868 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.683669    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-data\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.683868 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.683812    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istiod-ca-cert\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.683868 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.683847    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-workload-certs\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.685481 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.685456    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-envoy\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.685637 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.685618    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-podinfo\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.702919 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.702884    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-token\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.703245 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.703222    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr2b5\" (UniqueName: \"kubernetes.io/projected/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-kube-api-access-qr2b5\") pod \"e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") " pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:10.777742 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.777655    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:10.833446 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:10.833413    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:11.112666 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:11.112634    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"]
Sep 15 11:56:11.115126 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:56:11.115087    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedee2ae4_9151_4b76_9e8d_78cc89fe70ba.slice/crio-c22cbb2a2bbd3f84b7ecc5019f6a93e2901777ef40f625adb1bc9cfeefaad170 WatchSource:0}: Error finding container c22cbb2a2bbd3f84b7ecc5019f6a93e2901777ef40f625adb1bc9cfeefaad170: Status 404 returned error can't find the container with id c22cbb2a2bbd3f84b7ecc5019f6a93e2901777ef40f625adb1bc9cfeefaad170
Sep 15 11:56:11.117579 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:11.117533    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:56:11.117696 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:11.117636    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:56:11.117696 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:11.117686    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:56:11.279824 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:11.279771    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:11.559980 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:11.559882    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224" event={"ID":"edee2ae4-9151-4b76-9e8d-78cc89fe70ba","Type":"ContainerStarted","Data":"b949f522d916fa3982aa939ed0f82ec7925fc5d828821ac8421a5ab65bf5487e"}
Sep 15 11:56:11.559980 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:11.559939    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224" event={"ID":"edee2ae4-9151-4b76-9e8d-78cc89fe70ba","Type":"ContainerStarted","Data":"c22cbb2a2bbd3f84b7ecc5019f6a93e2901777ef40f625adb1bc9cfeefaad170"}
Sep 15 11:56:11.582833 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:11.582768    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224" podStartSLOduration=1.582752294 podStartE2EDuration="1.582752294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:56:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:56:11.580714961 +0000 UTC m=+1873.073492110" watchObservedRunningTime="2026-09-15 11:56:11.582752294 +0000 UTC m=+1873.075529441"
Sep 15 11:56:11.834515 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:11.834427    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:12.182528 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:12.182488    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:12.839646 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:12.839615    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:12.977212 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:12.977177    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:13.572027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:13.571995    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:13.572909 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:13.572874    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:13.573040 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:13.572967    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:13.671706 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:13.671672    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:13.781676 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:13.781642    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:14.490548 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:14.490510    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:14.718724 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:14.718671    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:15.580919 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:15.580880    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:16.308371 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:16.308338    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:16.408965 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:16.408923    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:16.774373 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:16.774339    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:16.864595 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:16.864534    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:16.907535 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:16.907502    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"]
Sep 15 11:56:16.912179 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:16.912159    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"
Sep 15 11:56:16.914873 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:16.914849    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-d40293-dockercfg-9wspb\""
Sep 15 11:56:16.915821 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:16.915802    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-d40293\""
Sep 15 11:56:16.924986 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:16.924965    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"]
Sep 15 11:56:16.947911 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:16.947878    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e47adc90-1857-47d3-9f60-546961e40c8a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5\" (UID: \"e47adc90-1857-47d3-9f60-546961e40c8a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"
Sep 15 11:56:16.948027 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:16.947966    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvvln\" (UniqueName: \"kubernetes.io/projected/e47adc90-1857-47d3-9f60-546961e40c8a-kube-api-access-tvvln\") pod \"payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5\" (UID: \"e47adc90-1857-47d3-9f60-546961e40c8a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"
Sep 15 11:56:16.951304 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:16.951277    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:17.048902 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:17.048821    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e47adc90-1857-47d3-9f60-546961e40c8a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5\" (UID: \"e47adc90-1857-47d3-9f60-546961e40c8a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"
Sep 15 11:56:17.048902 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:17.048870    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvvln\" (UniqueName: \"kubernetes.io/projected/e47adc90-1857-47d3-9f60-546961e40c8a-kube-api-access-tvvln\") pod \"payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5\" (UID: \"e47adc90-1857-47d3-9f60-546961e40c8a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"
Sep 15 11:56:17.049581 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:17.049541    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e47adc90-1857-47d3-9f60-546961e40c8a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5\" (UID: \"e47adc90-1857-47d3-9f60-546961e40c8a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"
Sep 15 11:56:17.056932 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:17.056907    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvvln\" (UniqueName: \"kubernetes.io/projected/e47adc90-1857-47d3-9f60-546961e40c8a-kube-api-access-tvvln\") pod \"payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5\" (UID: \"e47adc90-1857-47d3-9f60-546961e40c8a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"
Sep 15 11:56:17.225160 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:17.225127    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"
Sep 15 11:56:17.508694 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:17.508662    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"]
Sep 15 11:56:17.512785 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:56:17.512752    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode47adc90_1857_47d3_9f60_546961e40c8a.slice/crio-931acb79497520b00018914b952af453626ba07e9c09f5c7c72d6002f6be2311 WatchSource:0}: Error finding container 931acb79497520b00018914b952af453626ba07e9c09f5c7c72d6002f6be2311: Status 404 returned error can't find the container with id 931acb79497520b00018914b952af453626ba07e9c09f5c7c72d6002f6be2311
Sep 15 11:56:17.598195 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:17.598166    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5" event={"ID":"e47adc90-1857-47d3-9f60-546961e40c8a","Type":"ContainerStarted","Data":"9c955a0a913ef9b78129c14f940102d4baaaeac05c9722cff8d99888cd53bd63"}
Sep 15 11:56:17.598343 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:17.598223    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5" event={"ID":"e47adc90-1857-47d3-9f60-546961e40c8a","Type":"ContainerStarted","Data":"931acb79497520b00018914b952af453626ba07e9c09f5c7c72d6002f6be2311"}
Sep 15 11:56:17.598409 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:17.598353    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"
Sep 15 11:56:17.619614 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:17.619540    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5" podStartSLOduration=1.619519631 podStartE2EDuration="1.619519631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:56:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:56:17.616015157 +0000 UTC m=+1879.108792317" watchObservedRunningTime="2026-09-15 11:56:17.619519631 +0000 UTC m=+1879.112296781"
Sep 15 11:56:28.606717 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:28.606685    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"
Sep 15 11:56:29.612747 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:29.612702    2560 patch_prober.go:28] interesting pod/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 15 11:56:29.613148 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:29.612760    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224" podUID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 15 11:56:31.698151 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:31.698109    2560 generic.go:356] "Generic (PLEG): container finished" podID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerID="b949f522d916fa3982aa939ed0f82ec7925fc5d828821ac8421a5ab65bf5487e" exitCode=0
Sep 15 11:56:31.698598 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:31.698177    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224" event={"ID":"edee2ae4-9151-4b76-9e8d-78cc89fe70ba","Type":"ContainerDied","Data":"b949f522d916fa3982aa939ed0f82ec7925fc5d828821ac8421a5ab65bf5487e"}
Sep 15 11:56:31.698650 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:31.698616    2560 scope.go:117] "RemoveContainer" containerID="b949f522d916fa3982aa939ed0f82ec7925fc5d828821ac8421a5ab65bf5487e"
Sep 15 11:56:31.699790 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:31.699764    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:56:31.700003 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:31.699828    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:56:31.700003 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:31.699860    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:56:32.710322 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:32.710271    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224" event={"ID":"edee2ae4-9151-4b76-9e8d-78cc89fe70ba","Type":"ContainerStarted","Data":"e545594ef46edb8d03a036180852e10013719ab5a17e50ff2f78b27584a15cf9"}
Sep 15 11:56:32.834121 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:32.834087    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:32.834329 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:32.834305    2560 patch_prober.go:28] interesting pod/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 15 11:56:32.834387 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:32.834353    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224" podUID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 15 11:56:33.834033 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:33.834001    2560 patch_prober.go:28] interesting pod/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 15 11:56:33.834430 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:33.834054    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224" podUID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 15 11:56:34.834621 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:34.834583    2560 patch_prober.go:28] interesting pod/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 15 11:56:34.835022 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:34.834642    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224" podUID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 15 11:56:35.834530 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:35.834498    2560 patch_prober.go:28] interesting pod/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 15 11:56:35.834726 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:35.834550    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224" podUID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 15 11:56:36.833871 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:36.833831    2560 patch_prober.go:28] interesting pod/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 15 11:56:36.834065 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:36.833903    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224" podUID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 15 11:56:37.908328 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:37.908289    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:37.908905 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:37.908879    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:37.909840 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:37.909821    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:56:38.422200 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:38.422162    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:38.672046 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:38.672009    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:39.517763 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:39.517728    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:40.295443 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:40.295397    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:41.085759 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:41.085722    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:41.852626 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:41.852557    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:56:44.014726 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:56:44.014688    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5_e47adc90-1857-47d3-9f60-546961e40c8a/payload-processing/0.log"
Sep 15 11:57:02.980808 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:02.980773    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:05.305595 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:05.305530    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:35.584949 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:35.584922    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5_e47adc90-1857-47d3-9f60-546961e40c8a/payload-processing/0.log"
Sep 15 11:57:35.864058 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:35.864023    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:36.180756 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.180674    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"]
Sep 15 11:57:36.184364 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.184344    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.187182 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.187156    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-1e87f5-openshift-default-dockercfg-4vcnv\""
Sep 15 11:57:36.201532 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.201503    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"]
Sep 15 11:57:36.230690 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.230663    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/550f07c6-e31d-4f18-afed-369d6684700a-istio-token\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.230848 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.230705    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-istio-data\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.230848 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.230809    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-workload-socket\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.230978 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.230865    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/550f07c6-e31d-4f18-afed-369d6684700a-istio-podinfo\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.230978 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.230901    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kzfj\" (UniqueName: \"kubernetes.io/projected/550f07c6-e31d-4f18-afed-369d6684700a-kube-api-access-2kzfj\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.230978 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.230932    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-credential-socket\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.231138 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.231043    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-workload-certs\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.231138 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.231080    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-istio-envoy\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.231138 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.231113    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/550f07c6-e31d-4f18-afed-369d6684700a-istiod-ca-cert\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.237700 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.237655    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:36.310801 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.310347    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:36.334871 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.334834    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-istio-envoy\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.335143 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.335121    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/550f07c6-e31d-4f18-afed-369d6684700a-istiod-ca-cert\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.335348 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.335331    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/550f07c6-e31d-4f18-afed-369d6684700a-istio-token\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.335505 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.335485    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-istio-data\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.335954 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.335933    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-workload-socket\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.336119 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.336096    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/550f07c6-e31d-4f18-afed-369d6684700a-istiod-ca-cert\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.336209 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.335865    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-istio-data\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.336302 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.336283    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/550f07c6-e31d-4f18-afed-369d6684700a-istio-podinfo\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.336421 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.336404    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kzfj\" (UniqueName: \"kubernetes.io/projected/550f07c6-e31d-4f18-afed-369d6684700a-kube-api-access-2kzfj\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.336890 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.336868    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-credential-socket\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.337118 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.337099    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-workload-certs\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.337612 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.337591    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-workload-certs\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.337746 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.336625    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-workload-socket\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.338107 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.338087    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-credential-socket\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.338997 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.338974    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-istio-envoy\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.340892 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.340868    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/550f07c6-e31d-4f18-afed-369d6684700a-istio-podinfo\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.349067 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.349026    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/550f07c6-e31d-4f18-afed-369d6684700a-istio-token\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.355063 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.355036    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kzfj\" (UniqueName: \"kubernetes.io/projected/550f07c6-e31d-4f18-afed-369d6684700a-kube-api-access-2kzfj\") pod \"e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") " pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.495545 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.495451    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:36.794910 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.794883    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"]
Sep 15 11:57:36.797160 ip-10-0-138-153 kubenswrapper[2560]: W0915 11:57:36.797127    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod550f07c6_e31d_4f18_afed_369d6684700a.slice/crio-0af7058a82166ca3fc1e61f77d80d1a8404797508ca1e0a0c04bb18b46393540 WatchSource:0}: Error finding container 0af7058a82166ca3fc1e61f77d80d1a8404797508ca1e0a0c04bb18b46393540: Status 404 returned error can't find the container with id 0af7058a82166ca3fc1e61f77d80d1a8404797508ca1e0a0c04bb18b46393540
Sep 15 11:57:36.800020 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.799982    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:57:36.800108 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.800069    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:57:36.800149 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:36.800112    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:57:37.064319 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:37.064223    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j" event={"ID":"550f07c6-e31d-4f18-afed-369d6684700a","Type":"ContainerStarted","Data":"3c812278b1533d824be09ae7b14f6136306004b3cbcf9516f4e69be491dcbb57"}
Sep 15 11:57:37.064319 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:37.064274    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j" event={"ID":"550f07c6-e31d-4f18-afed-369d6684700a","Type":"ContainerStarted","Data":"0af7058a82166ca3fc1e61f77d80d1a8404797508ca1e0a0c04bb18b46393540"}
Sep 15 11:57:37.086960 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:37.086902    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j" podStartSLOduration=1.086881305 podStartE2EDuration="1.086881305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 11:57:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:57:37.08311107 +0000 UTC m=+1958.575888222" watchObservedRunningTime="2026-09-15 11:57:37.086881305 +0000 UTC m=+1958.579658454"
Sep 15 11:57:37.496455 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:37.496412    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:37.502500 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:37.502471    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:38.070190 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:38.070163    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:38.071118 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:38.071097    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:57:38.394846 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:38.394805    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:39.029993 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:39.029957    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:39.121268 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:39.121234    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:39.217874 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:39.217837    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:40.114897 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:40.114853    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:41.002594 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:41.002384    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:41.654883 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:41.654839    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:42.359593 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:42.359534    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:42.459379 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:42.459342    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:42.753702 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:42.753616    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:42.853192 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:42.853151    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:45.410701 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:45.410663    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:53.363579 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:53.363484    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:57:59.909630 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:59.909598    2560 scope.go:117] "RemoveContainer" containerID="41bab9d928c6ae739b628d83306966f83f3a5b714ebf2ea5770a4d57640b438f"
Sep 15 11:57:59.918501 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:57:59.918474    2560 scope.go:117] "RemoveContainer" containerID="897079ade6181e918738b2292ef1512f1c48be62a04f01cc7855927e96e18b30"
Sep 15 11:58:00.193957 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:00.193877    2560 generic.go:356] "Generic (PLEG): container finished" podID="550f07c6-e31d-4f18-afed-369d6684700a" containerID="3c812278b1533d824be09ae7b14f6136306004b3cbcf9516f4e69be491dcbb57" exitCode=0
Sep 15 11:58:00.194109 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:00.193956    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j" event={"ID":"550f07c6-e31d-4f18-afed-369d6684700a","Type":"ContainerDied","Data":"3c812278b1533d824be09ae7b14f6136306004b3cbcf9516f4e69be491dcbb57"}
Sep 15 11:58:00.194442 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:00.194426    2560 scope.go:117] "RemoveContainer" containerID="3c812278b1533d824be09ae7b14f6136306004b3cbcf9516f4e69be491dcbb57"
Sep 15 11:58:00.195693 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:00.195655    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:58:00.195818 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:00.195732    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:58:00.195818 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:00.195767    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 15 11:58:01.200766 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:01.200733    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j" event={"ID":"550f07c6-e31d-4f18-afed-369d6684700a","Type":"ContainerStarted","Data":"6860c8fc6df8021520151b2d53c903df203e5cf668eada296d1bb5771538524d"}
Sep 15 11:58:01.495875 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:01.495781    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:58:01.496039 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:01.495956    2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 15 11:58:01.496039 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:01.496000    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j" podUID="550f07c6-e31d-4f18-afed-369d6684700a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 15 11:58:02.496421 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:02.496382    2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 15 11:58:02.497100 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:02.497065    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j" podUID="550f07c6-e31d-4f18-afed-369d6684700a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 15 11:58:03.496542 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:03.496502    2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 15 11:58:03.497166 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:03.497136    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j" podUID="550f07c6-e31d-4f18-afed-369d6684700a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 15 11:58:04.496272 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:04.496227    2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 15 11:58:04.496494 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:04.496293    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j" podUID="550f07c6-e31d-4f18-afed-369d6684700a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 15 11:58:04.503401 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:04.503369    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:05.315367 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:05.315316    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:05.495994 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:05.495957    2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 15 11:58:05.496258 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:05.496229    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j" podUID="550f07c6-e31d-4f18-afed-369d6684700a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 15 11:58:06.212465 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:06.212437    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:06.495899 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:06.495805    2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 15 11:58:06.495899 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:06.495870    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j" podUID="550f07c6-e31d-4f18-afed-369d6684700a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 15 11:58:06.913811 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:06.913776    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:07.513177 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:07.513134    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:58:07.513686 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:07.513600    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:58:07.515423 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:07.515400    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:58:07.781514 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:07.781428    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:08.824360 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:08.824314    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:09.127677 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:09.127633    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:10.625303 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:10.625263    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:11.349814 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:11.349779    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:12.190773 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:12.190723    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:12.925386 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:12.925345    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:13.646858 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:13.646818    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:14.293790 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:14.293750    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:20.830019 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:20.829966    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:20.905116 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:20.905079    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:20.987019 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:20.986976    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:21.058380 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:21.058346    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:21.411881 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:21.411852    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:21.515417 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:21.515382    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:22.289602 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:22.289571    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:22.925146 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:22.925108    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:23.564669 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:23.564610    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:24.272769 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:24.272739    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:27.084727 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:27.084689    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:27.185388 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:27.185352    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:27.292736 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:27.292692    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"]
Sep 15 11:58:27.292981 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:27.292935    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j" podUID="550f07c6-e31d-4f18-afed-369d6684700a" containerName="istio-proxy" containerID="cri-o://6860c8fc6df8021520151b2d53c903df203e5cf668eada296d1bb5771538524d" gracePeriod=30
Sep 15 11:58:27.313861 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:27.313822    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:31.093656 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:31.093614    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:31.981740 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:31.981706    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:32.558210 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.558155    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:58:32.625626 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.625588    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-istio-envoy\") pod \"550f07c6-e31d-4f18-afed-369d6684700a\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") "
Sep 15 11:58:32.625820 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.625655    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kzfj\" (UniqueName: \"kubernetes.io/projected/550f07c6-e31d-4f18-afed-369d6684700a-kube-api-access-2kzfj\") pod \"550f07c6-e31d-4f18-afed-369d6684700a\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") "
Sep 15 11:58:32.625820 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.625680    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/550f07c6-e31d-4f18-afed-369d6684700a-istio-podinfo\") pod \"550f07c6-e31d-4f18-afed-369d6684700a\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") "
Sep 15 11:58:32.625820 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.625707    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/550f07c6-e31d-4f18-afed-369d6684700a-istio-token\") pod \"550f07c6-e31d-4f18-afed-369d6684700a\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") "
Sep 15 11:58:32.625820 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.625727    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-credential-socket\") pod \"550f07c6-e31d-4f18-afed-369d6684700a\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") "
Sep 15 11:58:32.625820 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.625760    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-workload-certs\") pod \"550f07c6-e31d-4f18-afed-369d6684700a\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") "
Sep 15 11:58:32.625820 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.625793    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-istio-data\") pod \"550f07c6-e31d-4f18-afed-369d6684700a\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") "
Sep 15 11:58:32.626162 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.625846    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-workload-socket\") pod \"550f07c6-e31d-4f18-afed-369d6684700a\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") "
Sep 15 11:58:32.626162 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.625903    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/550f07c6-e31d-4f18-afed-369d6684700a-istiod-ca-cert\") pod \"550f07c6-e31d-4f18-afed-369d6684700a\" (UID: \"550f07c6-e31d-4f18-afed-369d6684700a\") "
Sep 15 11:58:32.626162 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.626109    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "550f07c6-e31d-4f18-afed-369d6684700a" (UID: "550f07c6-e31d-4f18-afed-369d6684700a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:58:32.626162 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.626123    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "550f07c6-e31d-4f18-afed-369d6684700a" (UID: "550f07c6-e31d-4f18-afed-369d6684700a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:58:32.626379 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.626302    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:58:32.626379 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.626325    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:58:32.626379 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.626332    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "550f07c6-e31d-4f18-afed-369d6684700a" (UID: "550f07c6-e31d-4f18-afed-369d6684700a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:58:32.626551 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.626397    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/550f07c6-e31d-4f18-afed-369d6684700a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "550f07c6-e31d-4f18-afed-369d6684700a" (UID: "550f07c6-e31d-4f18-afed-369d6684700a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:58:32.626551 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.626469    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-istio-data" (OuterVolumeSpecName: "istio-data") pod "550f07c6-e31d-4f18-afed-369d6684700a" (UID: "550f07c6-e31d-4f18-afed-369d6684700a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:58:32.628022 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.627994    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/550f07c6-e31d-4f18-afed-369d6684700a-kube-api-access-2kzfj" (OuterVolumeSpecName: "kube-api-access-2kzfj") pod "550f07c6-e31d-4f18-afed-369d6684700a" (UID: "550f07c6-e31d-4f18-afed-369d6684700a"). InnerVolumeSpecName "kube-api-access-2kzfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:58:32.628022 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.628001    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/550f07c6-e31d-4f18-afed-369d6684700a-istio-token" (OuterVolumeSpecName: "istio-token") pod "550f07c6-e31d-4f18-afed-369d6684700a" (UID: "550f07c6-e31d-4f18-afed-369d6684700a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:58:32.628263 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.628237    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/550f07c6-e31d-4f18-afed-369d6684700a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "550f07c6-e31d-4f18-afed-369d6684700a" (UID: "550f07c6-e31d-4f18-afed-369d6684700a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:58:32.628263 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.628241    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "550f07c6-e31d-4f18-afed-369d6684700a" (UID: "550f07c6-e31d-4f18-afed-369d6684700a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:58:32.727358 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.727326    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:58:32.727358 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.727355    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:58:32.727537 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.727367    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/550f07c6-e31d-4f18-afed-369d6684700a-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:58:32.727537 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.727376    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/550f07c6-e31d-4f18-afed-369d6684700a-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:58:32.727537 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.727384    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kzfj\" (UniqueName: \"kubernetes.io/projected/550f07c6-e31d-4f18-afed-369d6684700a-kube-api-access-2kzfj\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:58:32.727537 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.727392    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/550f07c6-e31d-4f18-afed-369d6684700a-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:58:32.727537 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.727399    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/550f07c6-e31d-4f18-afed-369d6684700a-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:58:32.751042 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:32.751004    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:33.381723 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:33.381682    2560 generic.go:356] "Generic (PLEG): container finished" podID="550f07c6-e31d-4f18-afed-369d6684700a" containerID="6860c8fc6df8021520151b2d53c903df203e5cf668eada296d1bb5771538524d" exitCode=0
Sep 15 11:58:33.381911 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:33.381731    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"
Sep 15 11:58:33.381911 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:33.381765    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j" event={"ID":"550f07c6-e31d-4f18-afed-369d6684700a","Type":"ContainerDied","Data":"6860c8fc6df8021520151b2d53c903df203e5cf668eada296d1bb5771538524d"}
Sep 15 11:58:33.381911 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:33.381820    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j" event={"ID":"550f07c6-e31d-4f18-afed-369d6684700a","Type":"ContainerDied","Data":"0af7058a82166ca3fc1e61f77d80d1a8404797508ca1e0a0c04bb18b46393540"}
Sep 15 11:58:33.381911 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:33.381842    2560 scope.go:117] "RemoveContainer" containerID="6860c8fc6df8021520151b2d53c903df203e5cf668eada296d1bb5771538524d"
Sep 15 11:58:33.390060 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:33.390037    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:33.391548 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:33.391530    2560 scope.go:117] "RemoveContainer" containerID="3c812278b1533d824be09ae7b14f6136306004b3cbcf9516f4e69be491dcbb57"
Sep 15 11:58:33.400931 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:33.400906    2560 scope.go:117] "RemoveContainer" containerID="6860c8fc6df8021520151b2d53c903df203e5cf668eada296d1bb5771538524d"
Sep 15 11:58:33.401273 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:58:33.401247    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6860c8fc6df8021520151b2d53c903df203e5cf668eada296d1bb5771538524d\": container with ID starting with 6860c8fc6df8021520151b2d53c903df203e5cf668eada296d1bb5771538524d not found: ID does not exist" containerID="6860c8fc6df8021520151b2d53c903df203e5cf668eada296d1bb5771538524d"
Sep 15 11:58:33.401371 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:33.401284    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6860c8fc6df8021520151b2d53c903df203e5cf668eada296d1bb5771538524d"} err="failed to get container status \"6860c8fc6df8021520151b2d53c903df203e5cf668eada296d1bb5771538524d\": rpc error: code = NotFound desc = could not find container \"6860c8fc6df8021520151b2d53c903df203e5cf668eada296d1bb5771538524d\": container with ID starting with 6860c8fc6df8021520151b2d53c903df203e5cf668eada296d1bb5771538524d not found: ID does not exist"
Sep 15 11:58:33.401371 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:33.401306    2560 scope.go:117] "RemoveContainer" containerID="3c812278b1533d824be09ae7b14f6136306004b3cbcf9516f4e69be491dcbb57"
Sep 15 11:58:33.401646 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:58:33.401618    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c812278b1533d824be09ae7b14f6136306004b3cbcf9516f4e69be491dcbb57\": container with ID starting with 3c812278b1533d824be09ae7b14f6136306004b3cbcf9516f4e69be491dcbb57 not found: ID does not exist" containerID="3c812278b1533d824be09ae7b14f6136306004b3cbcf9516f4e69be491dcbb57"
Sep 15 11:58:33.401646 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:33.401644    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"]
Sep 15 11:58:33.401818 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:33.401652    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c812278b1533d824be09ae7b14f6136306004b3cbcf9516f4e69be491dcbb57"} err="failed to get container status \"3c812278b1533d824be09ae7b14f6136306004b3cbcf9516f4e69be491dcbb57\": rpc error: code = NotFound desc = could not find container \"3c812278b1533d824be09ae7b14f6136306004b3cbcf9516f4e69be491dcbb57\": container with ID starting with 3c812278b1533d824be09ae7b14f6136306004b3cbcf9516f4e69be491dcbb57 not found: ID does not exist"
Sep 15 11:58:33.405704 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:33.405682    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1e87f5-openshift-default-59fbc687bd-72b5j"]
Sep 15 11:58:34.030929 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:34.030893    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:34.755776 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:34.755743    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:35.035212 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:35.035114    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="550f07c6-e31d-4f18-afed-369d6684700a" path="/var/lib/kubelet/pods/550f07c6-e31d-4f18-afed-369d6684700a/volumes"
Sep 15 11:58:46.287355 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:46.287320    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:46.388485 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:46.388453    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:46.495628 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:46.495588    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:46.593318 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:46.593285    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:46.848065 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:46.847974    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:46.948172 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:46.948142    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:47.679421 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:47.679383    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:48.125686 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:48.125647    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:48.224901 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:48.224864    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:48.403676 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:48.403556    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:48.596531 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:48.596495    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:48.697503 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:48.697411    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:49.060261 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:49.059423    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:51.884291 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:51.884250    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:58:59.940875 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:58:59.940842    2560 scope.go:117] "RemoveContainer" containerID="89a297066ea32719085db705529e1aa308b61c4a82438e4c4811c234bacc8ea5"
Sep 15 11:59:27.981757 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:27.981665    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:28.778457 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:28.778417    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:34.893628 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:34.893593    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:34.993140 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:34.993102    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:35.161713 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:35.160026    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"]
Sep 15 11:59:35.161713 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:35.160358    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j" podUID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerName="istio-proxy" containerID="cri-o://411c8e8cbb3027e3ca86963d607a41506ee9e0c6d7a9c5e9c7891f5f3d86692a" gracePeriod=30
Sep 15 11:59:35.175457 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:35.175422    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:35.893439 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:35.893402    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:36.622728 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:36.622691    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:37.281498 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:37.281455    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:38.066193 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:38.066149    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:38.667537 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:38.667506    2560 patch_prober.go:28] interesting pod/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 11:59:38.667710 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:38.667556    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j" podUID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 11:59:38.722156 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:38.722113    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:40.418764 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.418736    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:59:40.562515 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.562430    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg9bf\" (UniqueName: \"kubernetes.io/projected/071de3b0-e4d5-43a0-b167-1356abd89f1b-kube-api-access-xg9bf\") pod \"071de3b0-e4d5-43a0-b167-1356abd89f1b\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") "
Sep 15 11:59:40.562515 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.562473    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-workload-socket\") pod \"071de3b0-e4d5-43a0-b167-1356abd89f1b\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") "
Sep 15 11:59:40.562515 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.562499    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-token\") pod \"071de3b0-e4d5-43a0-b167-1356abd89f1b\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") "
Sep 15 11:59:40.562831 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.562581    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-workload-certs\") pod \"071de3b0-e4d5-43a0-b167-1356abd89f1b\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") "
Sep 15 11:59:40.562831 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.562606    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-data\") pod \"071de3b0-e4d5-43a0-b167-1356abd89f1b\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") "
Sep 15 11:59:40.562831 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.562628    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-podinfo\") pod \"071de3b0-e4d5-43a0-b167-1356abd89f1b\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") "
Sep 15 11:59:40.562831 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.562669    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-credential-socket\") pod \"071de3b0-e4d5-43a0-b167-1356abd89f1b\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") "
Sep 15 11:59:40.562831 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.562699    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/071de3b0-e4d5-43a0-b167-1356abd89f1b-istiod-ca-cert\") pod \"071de3b0-e4d5-43a0-b167-1356abd89f1b\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") "
Sep 15 11:59:40.562831 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.562732    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-envoy\") pod \"071de3b0-e4d5-43a0-b167-1356abd89f1b\" (UID: \"071de3b0-e4d5-43a0-b167-1356abd89f1b\") "
Sep 15 11:59:40.562831 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.562762    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "071de3b0-e4d5-43a0-b167-1356abd89f1b" (UID: "071de3b0-e4d5-43a0-b167-1356abd89f1b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:59:40.563179 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.563006    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "071de3b0-e4d5-43a0-b167-1356abd89f1b" (UID: "071de3b0-e4d5-43a0-b167-1356abd89f1b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:59:40.563179 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.563040    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:40.563179 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.563028    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "071de3b0-e4d5-43a0-b167-1356abd89f1b" (UID: "071de3b0-e4d5-43a0-b167-1356abd89f1b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:59:40.563179 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.563062    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-data" (OuterVolumeSpecName: "istio-data") pod "071de3b0-e4d5-43a0-b167-1356abd89f1b" (UID: "071de3b0-e4d5-43a0-b167-1356abd89f1b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:59:40.563406 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.563275    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/071de3b0-e4d5-43a0-b167-1356abd89f1b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "071de3b0-e4d5-43a0-b167-1356abd89f1b" (UID: "071de3b0-e4d5-43a0-b167-1356abd89f1b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:59:40.564872 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.564840    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/071de3b0-e4d5-43a0-b167-1356abd89f1b-kube-api-access-xg9bf" (OuterVolumeSpecName: "kube-api-access-xg9bf") pod "071de3b0-e4d5-43a0-b167-1356abd89f1b" (UID: "071de3b0-e4d5-43a0-b167-1356abd89f1b"). InnerVolumeSpecName "kube-api-access-xg9bf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:59:40.564989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.564908    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-token" (OuterVolumeSpecName: "istio-token") pod "071de3b0-e4d5-43a0-b167-1356abd89f1b" (UID: "071de3b0-e4d5-43a0-b167-1356abd89f1b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:59:40.564989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.564914    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "071de3b0-e4d5-43a0-b167-1356abd89f1b" (UID: "071de3b0-e4d5-43a0-b167-1356abd89f1b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:59:40.564989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.564969    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "071de3b0-e4d5-43a0-b167-1356abd89f1b" (UID: "071de3b0-e4d5-43a0-b167-1356abd89f1b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:59:40.663958 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.663923    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/071de3b0-e4d5-43a0-b167-1356abd89f1b-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:40.663958 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.663948    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:40.663958 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.663957    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg9bf\" (UniqueName: \"kubernetes.io/projected/071de3b0-e4d5-43a0-b167-1356abd89f1b-kube-api-access-xg9bf\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:40.663958 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.663966    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:40.664253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.663975    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:40.664253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.663982    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:40.664253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.663990    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/071de3b0-e4d5-43a0-b167-1356abd89f1b-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:40.664253 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.663997    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/071de3b0-e4d5-43a0-b167-1356abd89f1b-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:40.746427 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.746395    2560 generic.go:356] "Generic (PLEG): container finished" podID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerID="411c8e8cbb3027e3ca86963d607a41506ee9e0c6d7a9c5e9c7891f5f3d86692a" exitCode=0
Sep 15 11:59:40.746608 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.746439    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"
Sep 15 11:59:40.746608 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.746448    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j" event={"ID":"071de3b0-e4d5-43a0-b167-1356abd89f1b","Type":"ContainerDied","Data":"411c8e8cbb3027e3ca86963d607a41506ee9e0c6d7a9c5e9c7891f5f3d86692a"}
Sep 15 11:59:40.746608 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.746515    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j" event={"ID":"071de3b0-e4d5-43a0-b167-1356abd89f1b","Type":"ContainerDied","Data":"f01425b171c197da004e6cca939cda7595e1694ed62d440dbcc47ae8b9f7b64f"}
Sep 15 11:59:40.746608 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.746538    2560 scope.go:117] "RemoveContainer" containerID="411c8e8cbb3027e3ca86963d607a41506ee9e0c6d7a9c5e9c7891f5f3d86692a"
Sep 15 11:59:40.761980 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.761958    2560 scope.go:117] "RemoveContainer" containerID="953301990e5c5e4cfab9aae26b197f19d83cff51aa46b4cef662dd35fedb0aaa"
Sep 15 11:59:40.770572 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.770541    2560 scope.go:117] "RemoveContainer" containerID="411c8e8cbb3027e3ca86963d607a41506ee9e0c6d7a9c5e9c7891f5f3d86692a"
Sep 15 11:59:40.770874 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:59:40.770855    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"411c8e8cbb3027e3ca86963d607a41506ee9e0c6d7a9c5e9c7891f5f3d86692a\": container with ID starting with 411c8e8cbb3027e3ca86963d607a41506ee9e0c6d7a9c5e9c7891f5f3d86692a not found: ID does not exist" containerID="411c8e8cbb3027e3ca86963d607a41506ee9e0c6d7a9c5e9c7891f5f3d86692a"
Sep 15 11:59:40.770939 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.770880    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"411c8e8cbb3027e3ca86963d607a41506ee9e0c6d7a9c5e9c7891f5f3d86692a"} err="failed to get container status \"411c8e8cbb3027e3ca86963d607a41506ee9e0c6d7a9c5e9c7891f5f3d86692a\": rpc error: code = NotFound desc = could not find container \"411c8e8cbb3027e3ca86963d607a41506ee9e0c6d7a9c5e9c7891f5f3d86692a\": container with ID starting with 411c8e8cbb3027e3ca86963d607a41506ee9e0c6d7a9c5e9c7891f5f3d86692a not found: ID does not exist"
Sep 15 11:59:40.770939 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.770896    2560 scope.go:117] "RemoveContainer" containerID="953301990e5c5e4cfab9aae26b197f19d83cff51aa46b4cef662dd35fedb0aaa"
Sep 15 11:59:40.771157 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:59:40.771136    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"953301990e5c5e4cfab9aae26b197f19d83cff51aa46b4cef662dd35fedb0aaa\": container with ID starting with 953301990e5c5e4cfab9aae26b197f19d83cff51aa46b4cef662dd35fedb0aaa not found: ID does not exist" containerID="953301990e5c5e4cfab9aae26b197f19d83cff51aa46b4cef662dd35fedb0aaa"
Sep 15 11:59:40.771276 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.771160    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"953301990e5c5e4cfab9aae26b197f19d83cff51aa46b4cef662dd35fedb0aaa"} err="failed to get container status \"953301990e5c5e4cfab9aae26b197f19d83cff51aa46b4cef662dd35fedb0aaa\": rpc error: code = NotFound desc = could not find container \"953301990e5c5e4cfab9aae26b197f19d83cff51aa46b4cef662dd35fedb0aaa\": container with ID starting with 953301990e5c5e4cfab9aae26b197f19d83cff51aa46b4cef662dd35fedb0aaa not found: ID does not exist"
Sep 15 11:59:40.773160 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.773092    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"]
Sep 15 11:59:40.775718 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:40.775699    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-173324-openshift-default-7cf5df54fd-46p9j"]
Sep 15 11:59:41.034772 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:41.034741    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="071de3b0-e4d5-43a0-b167-1356abd89f1b" path="/var/lib/kubelet/pods/071de3b0-e4d5-43a0-b167-1356abd89f1b/volumes"
Sep 15 11:59:46.013744 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.013707    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:46.042405 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.042376    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"]
Sep 15 11:59:46.042621 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.042593    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5" podUID="e47adc90-1857-47d3-9f60-546961e40c8a" containerName="payload-processing" containerID="cri-o://9c955a0a913ef9b78129c14f940102d4baaaeac05c9722cff8d99888cd53bd63" gracePeriod=30
Sep 15 11:59:46.338863 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.338832    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"
Sep 15 11:59:46.520437 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.520396    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvvln\" (UniqueName: \"kubernetes.io/projected/e47adc90-1857-47d3-9f60-546961e40c8a-kube-api-access-tvvln\") pod \"e47adc90-1857-47d3-9f60-546961e40c8a\" (UID: \"e47adc90-1857-47d3-9f60-546961e40c8a\") "
Sep 15 11:59:46.520668 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.520461    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e47adc90-1857-47d3-9f60-546961e40c8a-plugins-config-volume\") pod \"e47adc90-1857-47d3-9f60-546961e40c8a\" (UID: \"e47adc90-1857-47d3-9f60-546961e40c8a\") "
Sep 15 11:59:46.520907 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.520877    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e47adc90-1857-47d3-9f60-546961e40c8a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e47adc90-1857-47d3-9f60-546961e40c8a" (UID: "e47adc90-1857-47d3-9f60-546961e40c8a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:59:46.522646 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.522607    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e47adc90-1857-47d3-9f60-546961e40c8a-kube-api-access-tvvln" (OuterVolumeSpecName: "kube-api-access-tvvln") pod "e47adc90-1857-47d3-9f60-546961e40c8a" (UID: "e47adc90-1857-47d3-9f60-546961e40c8a"). InnerVolumeSpecName "kube-api-access-tvvln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:59:46.621927 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.621891    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvvln\" (UniqueName: \"kubernetes.io/projected/e47adc90-1857-47d3-9f60-546961e40c8a-kube-api-access-tvvln\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:46.621927 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.621923    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e47adc90-1857-47d3-9f60-546961e40c8a-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:46.779956 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.779916    2560 generic.go:356] "Generic (PLEG): container finished" podID="e47adc90-1857-47d3-9f60-546961e40c8a" containerID="9c955a0a913ef9b78129c14f940102d4baaaeac05c9722cff8d99888cd53bd63" exitCode=0
Sep 15 11:59:46.780129 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.779967    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5" event={"ID":"e47adc90-1857-47d3-9f60-546961e40c8a","Type":"ContainerDied","Data":"9c955a0a913ef9b78129c14f940102d4baaaeac05c9722cff8d99888cd53bd63"}
Sep 15 11:59:46.780129 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.779973    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"
Sep 15 11:59:46.780129 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.780040    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5" event={"ID":"e47adc90-1857-47d3-9f60-546961e40c8a","Type":"ContainerDied","Data":"931acb79497520b00018914b952af453626ba07e9c09f5c7c72d6002f6be2311"}
Sep 15 11:59:46.780129 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.780063    2560 scope.go:117] "RemoveContainer" containerID="9c955a0a913ef9b78129c14f940102d4baaaeac05c9722cff8d99888cd53bd63"
Sep 15 11:59:46.789709 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.789684    2560 scope.go:117] "RemoveContainer" containerID="9c955a0a913ef9b78129c14f940102d4baaaeac05c9722cff8d99888cd53bd63"
Sep 15 11:59:46.790039 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:59:46.790008    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c955a0a913ef9b78129c14f940102d4baaaeac05c9722cff8d99888cd53bd63\": container with ID starting with 9c955a0a913ef9b78129c14f940102d4baaaeac05c9722cff8d99888cd53bd63 not found: ID does not exist" containerID="9c955a0a913ef9b78129c14f940102d4baaaeac05c9722cff8d99888cd53bd63"
Sep 15 11:59:46.790151 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.790051    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c955a0a913ef9b78129c14f940102d4baaaeac05c9722cff8d99888cd53bd63"} err="failed to get container status \"9c955a0a913ef9b78129c14f940102d4baaaeac05c9722cff8d99888cd53bd63\": rpc error: code = NotFound desc = could not find container \"9c955a0a913ef9b78129c14f940102d4baaaeac05c9722cff8d99888cd53bd63\": container with ID starting with 9c955a0a913ef9b78129c14f940102d4baaaeac05c9722cff8d99888cd53bd63 not found: ID does not exist"
Sep 15 11:59:46.803406 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.803373    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"]
Sep 15 11:59:46.807461 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:46.807439    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d40293-7fdbcfc787-59hv5"]
Sep 15 11:59:47.035546 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:47.035460    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e47adc90-1857-47d3-9f60-546961e40c8a" path="/var/lib/kubelet/pods/e47adc90-1857-47d3-9f60-546961e40c8a/volumes"
Sep 15 11:59:47.794860 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:47.794820    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:47.898387 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:47.898349    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:48.282654 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:48.282619    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:48.970954 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:48.970905    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:52.469989 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:52.469951    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:52.570529 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:52.570484    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:52.763927 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:52.763850    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"]
Sep 15 11:59:52.764105 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:52.764082    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224" podUID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerName="istio-proxy" containerID="cri-o://e545594ef46edb8d03a036180852e10013719ab5a17e50ff2f78b27584a15cf9" gracePeriod=30
Sep 15 11:59:52.799771 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:52.799745    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:52.911534 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:52.911496    2560 patch_prober.go:28] interesting pod/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 11:59:52.911718 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:52.911586    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224" podUID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 11:59:53.371842 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:53.371802    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:54.181359 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:54.181323    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:54.956286 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:54.956250    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:55.748608 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:55.748555    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 11:59:58.013078 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.013053    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:59:58.048453 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.048390    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-workload-certs\") pod \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") "
Sep 15 11:59:58.048453 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.048422    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-envoy\") pod \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") "
Sep 15 11:59:58.048684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.048474    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istiod-ca-cert\") pod \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") "
Sep 15 11:59:58.048684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.048526    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-data\") pod \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") "
Sep 15 11:59:58.048684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.048552    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-token\") pod \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") "
Sep 15 11:59:58.048684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.048618    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-podinfo\") pod \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") "
Sep 15 11:59:58.048684 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.048650    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "edee2ae4-9151-4b76-9e8d-78cc89fe70ba" (UID: "edee2ae4-9151-4b76-9e8d-78cc89fe70ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:59:58.048951 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.048687    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-workload-socket\") pod \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") "
Sep 15 11:59:58.048951 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.048717    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qr2b5\" (UniqueName: \"kubernetes.io/projected/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-kube-api-access-qr2b5\") pod \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") "
Sep 15 11:59:58.048951 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.048748    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-credential-socket\") pod \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\" (UID: \"edee2ae4-9151-4b76-9e8d-78cc89fe70ba\") "
Sep 15 11:59:58.048951 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.048868    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-data" (OuterVolumeSpecName: "istio-data") pod "edee2ae4-9151-4b76-9e8d-78cc89fe70ba" (UID: "edee2ae4-9151-4b76-9e8d-78cc89fe70ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:59:58.048951 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.048909    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "edee2ae4-9151-4b76-9e8d-78cc89fe70ba" (UID: "edee2ae4-9151-4b76-9e8d-78cc89fe70ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:59:58.049214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.048999    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "edee2ae4-9151-4b76-9e8d-78cc89fe70ba" (UID: "edee2ae4-9151-4b76-9e8d-78cc89fe70ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:59:58.049214 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.049084    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "edee2ae4-9151-4b76-9e8d-78cc89fe70ba" (UID: "edee2ae4-9151-4b76-9e8d-78cc89fe70ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:59:58.049320 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.049260    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:58.049320 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.049277    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:58.049320 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.049290    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:58.049320 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.049303    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:58.049320 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.049316    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:58.050902 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.050872    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "edee2ae4-9151-4b76-9e8d-78cc89fe70ba" (UID: "edee2ae4-9151-4b76-9e8d-78cc89fe70ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 11:59:58.050902 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.050882    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-token" (OuterVolumeSpecName: "istio-token") pod "edee2ae4-9151-4b76-9e8d-78cc89fe70ba" (UID: "edee2ae4-9151-4b76-9e8d-78cc89fe70ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:59:58.051326 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.051298    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-kube-api-access-qr2b5" (OuterVolumeSpecName: "kube-api-access-qr2b5") pod "edee2ae4-9151-4b76-9e8d-78cc89fe70ba" (UID: "edee2ae4-9151-4b76-9e8d-78cc89fe70ba"). InnerVolumeSpecName "kube-api-access-qr2b5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:59:58.051434 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.051299    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "edee2ae4-9151-4b76-9e8d-78cc89fe70ba" (UID: "edee2ae4-9151-4b76-9e8d-78cc89fe70ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 11:59:58.150445 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.150420    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:58.150552 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.150449    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:58.150552 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.150460    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qr2b5\" (UniqueName: \"kubernetes.io/projected/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-kube-api-access-qr2b5\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:58.150552 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.150469    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edee2ae4-9151-4b76-9e8d-78cc89fe70ba-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 11:59:58.851064 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.851032    2560 generic.go:356] "Generic (PLEG): container finished" podID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerID="e545594ef46edb8d03a036180852e10013719ab5a17e50ff2f78b27584a15cf9" exitCode=0
Sep 15 11:59:58.851235 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.851091    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"
Sep 15 11:59:58.851235 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.851119    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224" event={"ID":"edee2ae4-9151-4b76-9e8d-78cc89fe70ba","Type":"ContainerDied","Data":"e545594ef46edb8d03a036180852e10013719ab5a17e50ff2f78b27584a15cf9"}
Sep 15 11:59:58.851235 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.851197    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224" event={"ID":"edee2ae4-9151-4b76-9e8d-78cc89fe70ba","Type":"ContainerDied","Data":"c22cbb2a2bbd3f84b7ecc5019f6a93e2901777ef40f625adb1bc9cfeefaad170"}
Sep 15 11:59:58.851235 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.851220    2560 scope.go:117] "RemoveContainer" containerID="e545594ef46edb8d03a036180852e10013719ab5a17e50ff2f78b27584a15cf9"
Sep 15 11:59:58.860848 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.860834    2560 scope.go:117] "RemoveContainer" containerID="b949f522d916fa3982aa939ed0f82ec7925fc5d828821ac8421a5ab65bf5487e"
Sep 15 11:59:58.868508 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.868492    2560 scope.go:117] "RemoveContainer" containerID="e545594ef46edb8d03a036180852e10013719ab5a17e50ff2f78b27584a15cf9"
Sep 15 11:59:58.868767 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:59:58.868750    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e545594ef46edb8d03a036180852e10013719ab5a17e50ff2f78b27584a15cf9\": container with ID starting with e545594ef46edb8d03a036180852e10013719ab5a17e50ff2f78b27584a15cf9 not found: ID does not exist" containerID="e545594ef46edb8d03a036180852e10013719ab5a17e50ff2f78b27584a15cf9"
Sep 15 11:59:58.868849 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.868773    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e545594ef46edb8d03a036180852e10013719ab5a17e50ff2f78b27584a15cf9"} err="failed to get container status \"e545594ef46edb8d03a036180852e10013719ab5a17e50ff2f78b27584a15cf9\": rpc error: code = NotFound desc = could not find container \"e545594ef46edb8d03a036180852e10013719ab5a17e50ff2f78b27584a15cf9\": container with ID starting with e545594ef46edb8d03a036180852e10013719ab5a17e50ff2f78b27584a15cf9 not found: ID does not exist"
Sep 15 11:59:58.868849 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.868788    2560 scope.go:117] "RemoveContainer" containerID="b949f522d916fa3982aa939ed0f82ec7925fc5d828821ac8421a5ab65bf5487e"
Sep 15 11:59:58.869019 ip-10-0-138-153 kubenswrapper[2560]: E0915 11:59:58.869004    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b949f522d916fa3982aa939ed0f82ec7925fc5d828821ac8421a5ab65bf5487e\": container with ID starting with b949f522d916fa3982aa939ed0f82ec7925fc5d828821ac8421a5ab65bf5487e not found: ID does not exist" containerID="b949f522d916fa3982aa939ed0f82ec7925fc5d828821ac8421a5ab65bf5487e"
Sep 15 11:59:58.869059 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.869022    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b949f522d916fa3982aa939ed0f82ec7925fc5d828821ac8421a5ab65bf5487e"} err="failed to get container status \"b949f522d916fa3982aa939ed0f82ec7925fc5d828821ac8421a5ab65bf5487e\": rpc error: code = NotFound desc = could not find container \"b949f522d916fa3982aa939ed0f82ec7925fc5d828821ac8421a5ab65bf5487e\": container with ID starting with b949f522d916fa3982aa939ed0f82ec7925fc5d828821ac8421a5ab65bf5487e not found: ID does not exist"
Sep 15 11:59:58.872931 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.872906    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"]
Sep 15 11:59:58.877105 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:58.877080    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-d40293-openshift-default-54c84b46b8-wl224"]
Sep 15 11:59:59.035348 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:59.035315    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" path="/var/lib/kubelet/pods/edee2ae4-9151-4b76-9e8d-78cc89fe70ba/volumes"
Sep 15 11:59:59.710097 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:59.710069    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 11:59:59.716205 ip-10-0-138-153 kubenswrapper[2560]: I0915 11:59:59.716182    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 12:00:00.135104 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.135076    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-2cc790-29824560-hndp6"]
Sep 15 12:00:00.135490 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.135477    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerName="istio-proxy"
Sep 15 12:00:00.135533 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.135495    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerName="istio-proxy"
Sep 15 12:00:00.135533 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.135507    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerName="istio-proxy"
Sep 15 12:00:00.135533 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.135513    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerName="istio-proxy"
Sep 15 12:00:00.135533 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.135528    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="550f07c6-e31d-4f18-afed-369d6684700a" containerName="istio-proxy"
Sep 15 12:00:00.135692 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.135536    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="550f07c6-e31d-4f18-afed-369d6684700a" containerName="istio-proxy"
Sep 15 12:00:00.135692 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.135549    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e47adc90-1857-47d3-9f60-546961e40c8a" containerName="payload-processing"
Sep 15 12:00:00.135692 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.135554    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e47adc90-1857-47d3-9f60-546961e40c8a" containerName="payload-processing"
Sep 15 12:00:00.135692 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.135630    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerName="istio-proxy"
Sep 15 12:00:00.135692 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.135638    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerName="istio-proxy"
Sep 15 12:00:00.135692 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.135644    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e47adc90-1857-47d3-9f60-546961e40c8a" containerName="payload-processing"
Sep 15 12:00:00.135692 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.135655    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="550f07c6-e31d-4f18-afed-369d6684700a" containerName="istio-proxy"
Sep 15 12:00:00.135692 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.135665    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerName="istio-proxy"
Sep 15 12:00:00.135692 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.135672    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="550f07c6-e31d-4f18-afed-369d6684700a" containerName="istio-proxy"
Sep 15 12:00:00.140659 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.140641    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-2cc790-29824560-hndp6"
Sep 15 12:00:00.142928 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.142909    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbxjk\""
Sep 15 12:00:00.151837 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.151683    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-2cc790-29824560-hndp6"
Sep 15 12:00:00.153716 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.153696    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-2cc790-29824560-hndp6"]
Sep 15 12:00:00.422267 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.422244    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-2cc790-29824560-hndp6"]
Sep 15 12:00:00.424120 ip-10-0-138-153 kubenswrapper[2560]: W0915 12:00:00.424084    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda31857_2d1b_42f5_b873_c6e37f9ce119.slice/crio-e5398c5bf1594fe1fe93b307f3f7d17dd29a85dc34d12cb4dff3b339765d869c WatchSource:0}: Error finding container e5398c5bf1594fe1fe93b307f3f7d17dd29a85dc34d12cb4dff3b339765d869c: Status 404 returned error can't find the container with id e5398c5bf1594fe1fe93b307f3f7d17dd29a85dc34d12cb4dff3b339765d869c
Sep 15 12:00:00.681630 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.681538    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:00.865516 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.865484    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-2cc790-29824560-hndp6" event={"ID":"bda31857-2d1b-42f5-b873-c6e37f9ce119","Type":"ContainerStarted","Data":"90ac9785fd7f935e5ccf6f3f64540004e66f62f4326bc454517a6a9d49491fa8"}
Sep 15 12:00:00.865723 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.865535    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-2cc790-29824560-hndp6" event={"ID":"bda31857-2d1b-42f5-b873-c6e37f9ce119","Type":"ContainerStarted","Data":"e5398c5bf1594fe1fe93b307f3f7d17dd29a85dc34d12cb4dff3b339765d869c"}
Sep 15 12:00:00.881549 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:00.881498    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-2cc790-29824560-hndp6" podStartSLOduration=0.881482073 podStartE2EDuration="881.482073ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:00:00.879804767 +0000 UTC m=+2102.372581914" watchObservedRunningTime="2026-09-15 12:00:00.881482073 +0000 UTC m=+2102.374259220"
Sep 15 12:00:01.872095 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:01.872061    2560 generic.go:356] "Generic (PLEG): container finished" podID="bda31857-2d1b-42f5-b873-c6e37f9ce119" containerID="90ac9785fd7f935e5ccf6f3f64540004e66f62f4326bc454517a6a9d49491fa8" exitCode=0
Sep 15 12:00:01.872486 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:01.872138    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-2cc790-29824560-hndp6" event={"ID":"bda31857-2d1b-42f5-b873-c6e37f9ce119","Type":"ContainerDied","Data":"90ac9785fd7f935e5ccf6f3f64540004e66f62f4326bc454517a6a9d49491fa8"}
Sep 15 12:00:03.007245 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:03.007222    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-2cc790-29824560-hndp6"
Sep 15 12:00:03.708089 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:03.708052    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:03.721492 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:03.721458    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"]
Sep 15 12:00:03.721771 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:03.721742    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4" podUID="12ca1834-cc47-44d1-9af2-91d5d92e4cc4" containerName="payload-processing" containerID="cri-o://58d19ab575bcf0d3e4636f503f58983b081824542d858cb14cbde0335ecbd1e6" gracePeriod=30
Sep 15 12:00:03.778468 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:03.778420    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:03.886280 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:03.886249    2560 generic.go:356] "Generic (PLEG): container finished" podID="12ca1834-cc47-44d1-9af2-91d5d92e4cc4" containerID="58d19ab575bcf0d3e4636f503f58983b081824542d858cb14cbde0335ecbd1e6" exitCode=0
Sep 15 12:00:03.886441 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:03.886324    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4" event={"ID":"12ca1834-cc47-44d1-9af2-91d5d92e4cc4","Type":"ContainerDied","Data":"58d19ab575bcf0d3e4636f503f58983b081824542d858cb14cbde0335ecbd1e6"}
Sep 15 12:00:03.890019 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:03.887940    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-2cc790-29824560-hndp6" event={"ID":"bda31857-2d1b-42f5-b873-c6e37f9ce119","Type":"ContainerDied","Data":"e5398c5bf1594fe1fe93b307f3f7d17dd29a85dc34d12cb4dff3b339765d869c"}
Sep 15 12:00:03.890019 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:03.888006    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5398c5bf1594fe1fe93b307f3f7d17dd29a85dc34d12cb4dff3b339765d869c"
Sep 15 12:00:03.890019 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:03.888075    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-2cc790-29824560-hndp6"
Sep 15 12:00:03.994539 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:03.994510    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"
Sep 15 12:00:04.081607 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:04.081555    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:04.104618 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:04.104583    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4pnk\" (UniqueName: \"kubernetes.io/projected/12ca1834-cc47-44d1-9af2-91d5d92e4cc4-kube-api-access-j4pnk\") pod \"12ca1834-cc47-44d1-9af2-91d5d92e4cc4\" (UID: \"12ca1834-cc47-44d1-9af2-91d5d92e4cc4\") "
Sep 15 12:00:04.104802 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:04.104633    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ca1834-cc47-44d1-9af2-91d5d92e4cc4-plugins-config-volume\") pod \"12ca1834-cc47-44d1-9af2-91d5d92e4cc4\" (UID: \"12ca1834-cc47-44d1-9af2-91d5d92e4cc4\") "
Sep 15 12:00:04.105087 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:04.105053    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12ca1834-cc47-44d1-9af2-91d5d92e4cc4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "12ca1834-cc47-44d1-9af2-91d5d92e4cc4" (UID: "12ca1834-cc47-44d1-9af2-91d5d92e4cc4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:00:04.106836 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:04.106801    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12ca1834-cc47-44d1-9af2-91d5d92e4cc4-kube-api-access-j4pnk" (OuterVolumeSpecName: "kube-api-access-j4pnk") pod "12ca1834-cc47-44d1-9af2-91d5d92e4cc4" (UID: "12ca1834-cc47-44d1-9af2-91d5d92e4cc4"). InnerVolumeSpecName "kube-api-access-j4pnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:00:04.205638 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:04.205605    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4pnk\" (UniqueName: \"kubernetes.io/projected/12ca1834-cc47-44d1-9af2-91d5d92e4cc4-kube-api-access-j4pnk\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:04.205638 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:04.205632    2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ca1834-cc47-44d1-9af2-91d5d92e4cc4-plugins-config-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:04.293196 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:04.292866    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:04.487856 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:04.487817    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:04.633041 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:04.633003    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:04.894170 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:04.894095    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"
Sep 15 12:00:04.894349 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:04.894085    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4" event={"ID":"12ca1834-cc47-44d1-9af2-91d5d92e4cc4","Type":"ContainerDied","Data":"0242210f0b965dcab146816a970e85e9e0f734c5b559ac3ec7977f70d40ba741"}
Sep 15 12:00:04.894349 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:04.894242    2560 scope.go:117] "RemoveContainer" containerID="58d19ab575bcf0d3e4636f503f58983b081824542d858cb14cbde0335ecbd1e6"
Sep 15 12:00:04.919309 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:04.919279    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"]
Sep 15 12:00:04.923824 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:04.923796    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7efa8b-5bd6d54d9c-rfrf4"]
Sep 15 12:00:05.034024 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:05.033993    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12ca1834-cc47-44d1-9af2-91d5d92e4cc4" path="/var/lib/kubelet/pods/12ca1834-cc47-44d1-9af2-91d5d92e4cc4/volumes"
Sep 15 12:00:05.274983 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:05.274894    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:10.248841 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:10.248790    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:10.346643 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:10.346608    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:10.425776 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:10.425744    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"]
Sep 15 12:00:10.425982 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:10.425956    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerName="istio-proxy" containerID="cri-o://5e9b930e1ea49a4205e8634466411d3f46d4a67e754d330ce0200341757639d0" gracePeriod=30
Sep 15 12:00:11.080538 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:11.080499    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:11.182187 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:11.182150    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:11.857523 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:11.857493    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:12.509083 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:12.509051    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:15.688119 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.688093    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 12:00:15.815133 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815056    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfjtq\" (UniqueName: \"kubernetes.io/projected/7e31b7b9-6f7b-403f-b310-551df113f4bb-kube-api-access-lfjtq\") pod \"7e31b7b9-6f7b-403f-b310-551df113f4bb\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") "
Sep 15 12:00:15.815133 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815107    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-token\") pod \"7e31b7b9-6f7b-403f-b310-551df113f4bb\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") "
Sep 15 12:00:15.815334 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815145    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-credential-socket\") pod \"7e31b7b9-6f7b-403f-b310-551df113f4bb\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") "
Sep 15 12:00:15.815334 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815200    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-envoy\") pod \"7e31b7b9-6f7b-403f-b310-551df113f4bb\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") "
Sep 15 12:00:15.815334 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815227    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-workload-socket\") pod \"7e31b7b9-6f7b-403f-b310-551df113f4bb\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") "
Sep 15 12:00:15.815334 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815251    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-workload-certs\") pod \"7e31b7b9-6f7b-403f-b310-551df113f4bb\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") "
Sep 15 12:00:15.815334 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815281    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-podinfo\") pod \"7e31b7b9-6f7b-403f-b310-551df113f4bb\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") "
Sep 15 12:00:15.815621 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815337    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-data\") pod \"7e31b7b9-6f7b-403f-b310-551df113f4bb\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") "
Sep 15 12:00:15.815621 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815378    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e31b7b9-6f7b-403f-b310-551df113f4bb-istiod-ca-cert\") pod \"7e31b7b9-6f7b-403f-b310-551df113f4bb\" (UID: \"7e31b7b9-6f7b-403f-b310-551df113f4bb\") "
Sep 15 12:00:15.815621 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815477    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7e31b7b9-6f7b-403f-b310-551df113f4bb" (UID: "7e31b7b9-6f7b-403f-b310-551df113f4bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:00:15.815621 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815525    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7e31b7b9-6f7b-403f-b310-551df113f4bb" (UID: "7e31b7b9-6f7b-403f-b310-551df113f4bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:00:15.815901 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815780    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-data" (OuterVolumeSpecName: "istio-data") pod "7e31b7b9-6f7b-403f-b310-551df113f4bb" (UID: "7e31b7b9-6f7b-403f-b310-551df113f4bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:00:15.815901 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815816    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:15.815901 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815840    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:15.815901 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815875    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e31b7b9-6f7b-403f-b310-551df113f4bb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7e31b7b9-6f7b-403f-b310-551df113f4bb" (UID: "7e31b7b9-6f7b-403f-b310-551df113f4bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:00:15.816091 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.815928    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7e31b7b9-6f7b-403f-b310-551df113f4bb" (UID: "7e31b7b9-6f7b-403f-b310-551df113f4bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:00:15.817417 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.817393    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e31b7b9-6f7b-403f-b310-551df113f4bb-kube-api-access-lfjtq" (OuterVolumeSpecName: "kube-api-access-lfjtq") pod "7e31b7b9-6f7b-403f-b310-551df113f4bb" (UID: "7e31b7b9-6f7b-403f-b310-551df113f4bb"). InnerVolumeSpecName "kube-api-access-lfjtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:00:15.817548 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.817416    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-token" (OuterVolumeSpecName: "istio-token") pod "7e31b7b9-6f7b-403f-b310-551df113f4bb" (UID: "7e31b7b9-6f7b-403f-b310-551df113f4bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:00:15.817816 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.817789    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7e31b7b9-6f7b-403f-b310-551df113f4bb" (UID: "7e31b7b9-6f7b-403f-b310-551df113f4bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:00:15.817816 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.817802    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7e31b7b9-6f7b-403f-b310-551df113f4bb" (UID: "7e31b7b9-6f7b-403f-b310-551df113f4bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:00:15.916933 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.916906    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:15.916933 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.916931    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e31b7b9-6f7b-403f-b310-551df113f4bb-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:15.917090 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.916941    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfjtq\" (UniqueName: \"kubernetes.io/projected/7e31b7b9-6f7b-403f-b310-551df113f4bb-kube-api-access-lfjtq\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:15.917090 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.916950    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:15.917090 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.916958    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:15.917090 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.916966    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e31b7b9-6f7b-403f-b310-551df113f4bb-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:15.917090 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.916974    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e31b7b9-6f7b-403f-b310-551df113f4bb-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:15.953845 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.953815    2560 generic.go:356] "Generic (PLEG): container finished" podID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerID="5e9b930e1ea49a4205e8634466411d3f46d4a67e754d330ce0200341757639d0" exitCode=0
Sep 15 12:00:15.953985 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.953868    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" event={"ID":"7e31b7b9-6f7b-403f-b310-551df113f4bb","Type":"ContainerDied","Data":"5e9b930e1ea49a4205e8634466411d3f46d4a67e754d330ce0200341757639d0"}
Sep 15 12:00:15.953985 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.953881    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"
Sep 15 12:00:15.953985 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.953941    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz" event={"ID":"7e31b7b9-6f7b-403f-b310-551df113f4bb","Type":"ContainerDied","Data":"740bb576c2ea2f39fe0eb5108670e3af3bae50319ffa95b760437ea2c5959a2b"}
Sep 15 12:00:15.953985 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.953975    2560 scope.go:117] "RemoveContainer" containerID="5e9b930e1ea49a4205e8634466411d3f46d4a67e754d330ce0200341757639d0"
Sep 15 12:00:15.962621 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.962596    2560 scope.go:117] "RemoveContainer" containerID="a352c3a93423fef478a7626e7ff220044df81328757af11bd1bb88a59ee8592e"
Sep 15 12:00:15.970718 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.970701    2560 scope.go:117] "RemoveContainer" containerID="5e9b930e1ea49a4205e8634466411d3f46d4a67e754d330ce0200341757639d0"
Sep 15 12:00:15.970997 ip-10-0-138-153 kubenswrapper[2560]: E0915 12:00:15.970978    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e9b930e1ea49a4205e8634466411d3f46d4a67e754d330ce0200341757639d0\": container with ID starting with 5e9b930e1ea49a4205e8634466411d3f46d4a67e754d330ce0200341757639d0 not found: ID does not exist" containerID="5e9b930e1ea49a4205e8634466411d3f46d4a67e754d330ce0200341757639d0"
Sep 15 12:00:15.971061 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.971005    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e9b930e1ea49a4205e8634466411d3f46d4a67e754d330ce0200341757639d0"} err="failed to get container status \"5e9b930e1ea49a4205e8634466411d3f46d4a67e754d330ce0200341757639d0\": rpc error: code = NotFound desc = could not find container \"5e9b930e1ea49a4205e8634466411d3f46d4a67e754d330ce0200341757639d0\": container with ID starting with 5e9b930e1ea49a4205e8634466411d3f46d4a67e754d330ce0200341757639d0 not found: ID does not exist"
Sep 15 12:00:15.971061 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.971023    2560 scope.go:117] "RemoveContainer" containerID="a352c3a93423fef478a7626e7ff220044df81328757af11bd1bb88a59ee8592e"
Sep 15 12:00:15.971289 ip-10-0-138-153 kubenswrapper[2560]: E0915 12:00:15.971271    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a352c3a93423fef478a7626e7ff220044df81328757af11bd1bb88a59ee8592e\": container with ID starting with a352c3a93423fef478a7626e7ff220044df81328757af11bd1bb88a59ee8592e not found: ID does not exist" containerID="a352c3a93423fef478a7626e7ff220044df81328757af11bd1bb88a59ee8592e"
Sep 15 12:00:15.971335 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.971298    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a352c3a93423fef478a7626e7ff220044df81328757af11bd1bb88a59ee8592e"} err="failed to get container status \"a352c3a93423fef478a7626e7ff220044df81328757af11bd1bb88a59ee8592e\": rpc error: code = NotFound desc = could not find container \"a352c3a93423fef478a7626e7ff220044df81328757af11bd1bb88a59ee8592e\": container with ID starting with a352c3a93423fef478a7626e7ff220044df81328757af11bd1bb88a59ee8592e not found: ID does not exist"
Sep 15 12:00:15.976424 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.976394    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"]
Sep 15 12:00:15.980148 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:15.980129    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7efa8b-openshift-default-59b66b7c4-jq8kz"]
Sep 15 12:00:17.034551 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:17.034516    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" path="/var/lib/kubelet/pods/7e31b7b9-6f7b-403f-b310-551df113f4bb/volumes"
Sep 15 12:00:21.268942 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:21.268903    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-2cc790-29824560-hndp6"]
Sep 15 12:00:21.271020 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:21.270994    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-2cc790-29824560-hndp6"]
Sep 15 12:00:21.983575 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:21.983523    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:22.088394 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:22.088356    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:23.033974 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:23.033936    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bda31857-2d1b-42f5-b873-c6e37f9ce119" path="/var/lib/kubelet/pods/bda31857-2d1b-42f5-b873-c6e37f9ce119/volumes"
Sep 15 12:00:28.063080 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:28.063048    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"]
Sep 15 12:00:28.063595 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:28.063266    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj" podUID="4f9585cf-0d91-4424-946f-baf80cd5ca35" containerName="istio-proxy" containerID="cri-o://7e00fe2567e160ec018bebae171ff207b363804f2692279f9dc5f3ba9e9d974d" gracePeriod=30
Sep 15 12:00:30.721601 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:30.721546    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c8fb6dfc5-q6nnf"]
Sep 15 12:00:30.722133 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:30.721786    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf" podUID="9a050db2-5725-4e16-97b0-1da533d92eee" containerName="manager" containerID="cri-o://9dfd33dbdec7fb2da3d8071eb83df1ab7cd300ba266a8c0360b5b951f6f5aa64" gracePeriod=30
Sep 15 12:00:30.905119 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:30.905082    2560 patch_prober.go:28] interesting pod/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 12:00:30.905339 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:30.905146    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj" podUID="4f9585cf-0d91-4424-946f-baf80cd5ca35" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 12:00:30.970403 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:30.970374    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 12:00:31.035475 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.035402    2560 generic.go:356] "Generic (PLEG): container finished" podID="9a050db2-5725-4e16-97b0-1da533d92eee" containerID="9dfd33dbdec7fb2da3d8071eb83df1ab7cd300ba266a8c0360b5b951f6f5aa64" exitCode=0
Sep 15 12:00:31.035646 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.035501    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf"
Sep 15 12:00:31.036850 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.036830    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf" event={"ID":"9a050db2-5725-4e16-97b0-1da533d92eee","Type":"ContainerDied","Data":"9dfd33dbdec7fb2da3d8071eb83df1ab7cd300ba266a8c0360b5b951f6f5aa64"}
Sep 15 12:00:31.036929 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.036868    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c8fb6dfc5-q6nnf" event={"ID":"9a050db2-5725-4e16-97b0-1da533d92eee","Type":"ContainerDied","Data":"0f3bfce466aa13eac248ba74dea4af8c1c38fc6a544216865cb93a4ec0b7c858"}
Sep 15 12:00:31.036929 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.036889    2560 scope.go:117] "RemoveContainer" containerID="9dfd33dbdec7fb2da3d8071eb83df1ab7cd300ba266a8c0360b5b951f6f5aa64"
Sep 15 12:00:31.046472 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.046451    2560 scope.go:117] "RemoveContainer" containerID="9dfd33dbdec7fb2da3d8071eb83df1ab7cd300ba266a8c0360b5b951f6f5aa64"
Sep 15 12:00:31.046783 ip-10-0-138-153 kubenswrapper[2560]: E0915 12:00:31.046760    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dfd33dbdec7fb2da3d8071eb83df1ab7cd300ba266a8c0360b5b951f6f5aa64\": container with ID starting with 9dfd33dbdec7fb2da3d8071eb83df1ab7cd300ba266a8c0360b5b951f6f5aa64 not found: ID does not exist" containerID="9dfd33dbdec7fb2da3d8071eb83df1ab7cd300ba266a8c0360b5b951f6f5aa64"
Sep 15 12:00:31.046856 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.046791    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dfd33dbdec7fb2da3d8071eb83df1ab7cd300ba266a8c0360b5b951f6f5aa64"} err="failed to get container status \"9dfd33dbdec7fb2da3d8071eb83df1ab7cd300ba266a8c0360b5b951f6f5aa64\": rpc error: code = NotFound desc = could not find container \"9dfd33dbdec7fb2da3d8071eb83df1ab7cd300ba266a8c0360b5b951f6f5aa64\": container with ID starting with 9dfd33dbdec7fb2da3d8071eb83df1ab7cd300ba266a8c0360b5b951f6f5aa64 not found: ID does not exist"
Sep 15 12:00:31.057101 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.057082    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a050db2-5725-4e16-97b0-1da533d92eee-metrics-certs\") pod \"9a050db2-5725-4e16-97b0-1da533d92eee\" (UID: \"9a050db2-5725-4e16-97b0-1da533d92eee\") "
Sep 15 12:00:31.057295 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.057274    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mbr7\" (UniqueName: \"kubernetes.io/projected/9a050db2-5725-4e16-97b0-1da533d92eee-kube-api-access-8mbr7\") pod \"9a050db2-5725-4e16-97b0-1da533d92eee\" (UID: \"9a050db2-5725-4e16-97b0-1da533d92eee\") "
Sep 15 12:00:31.057366 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.057310    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a050db2-5725-4e16-97b0-1da533d92eee-webhook-cert\") pod \"9a050db2-5725-4e16-97b0-1da533d92eee\" (UID: \"9a050db2-5725-4e16-97b0-1da533d92eee\") "
Sep 15 12:00:31.059100 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.059070    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a050db2-5725-4e16-97b0-1da533d92eee-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9a050db2-5725-4e16-97b0-1da533d92eee" (UID: "9a050db2-5725-4e16-97b0-1da533d92eee"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:00:31.059311 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.059290    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a050db2-5725-4e16-97b0-1da533d92eee-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9a050db2-5725-4e16-97b0-1da533d92eee" (UID: "9a050db2-5725-4e16-97b0-1da533d92eee"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:00:31.059363 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.059342    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a050db2-5725-4e16-97b0-1da533d92eee-kube-api-access-8mbr7" (OuterVolumeSpecName: "kube-api-access-8mbr7") pod "9a050db2-5725-4e16-97b0-1da533d92eee" (UID: "9a050db2-5725-4e16-97b0-1da533d92eee"). InnerVolumeSpecName "kube-api-access-8mbr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:00:31.099541 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.099491    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:31.158203 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.158178    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mbr7\" (UniqueName: \"kubernetes.io/projected/9a050db2-5725-4e16-97b0-1da533d92eee-kube-api-access-8mbr7\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:31.158203 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.158201    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a050db2-5725-4e16-97b0-1da533d92eee-webhook-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:31.158203 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.158210    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a050db2-5725-4e16-97b0-1da533d92eee-metrics-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:31.202398 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.202371    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:00:31.358167 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.358137    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c8fb6dfc5-q6nnf"]
Sep 15 12:00:31.363346 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:31.363323    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5c8fb6dfc5-q6nnf"]
Sep 15 12:00:33.033958 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.033917    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a050db2-5725-4e16-97b0-1da533d92eee" path="/var/lib/kubelet/pods/9a050db2-5725-4e16-97b0-1da533d92eee/volumes"
Sep 15 12:00:33.325635 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.325609    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 12:00:33.376422 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.376389    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7csr\" (UniqueName: \"kubernetes.io/projected/4f9585cf-0d91-4424-946f-baf80cd5ca35-kube-api-access-q7csr\") pod \"4f9585cf-0d91-4424-946f-baf80cd5ca35\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") "
Sep 15 12:00:33.376624 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.376440    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-data\") pod \"4f9585cf-0d91-4424-946f-baf80cd5ca35\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") "
Sep 15 12:00:33.376624 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.376600    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-envoy\") pod \"4f9585cf-0d91-4424-946f-baf80cd5ca35\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") "
Sep 15 12:00:33.376758 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.376664    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-token\") pod \"4f9585cf-0d91-4424-946f-baf80cd5ca35\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") "
Sep 15 12:00:33.376758 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.376705    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-workload-certs\") pod \"4f9585cf-0d91-4424-946f-baf80cd5ca35\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") "
Sep 15 12:00:33.376758 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.376748    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-podinfo\") pod \"4f9585cf-0d91-4424-946f-baf80cd5ca35\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") "
Sep 15 12:00:33.376929 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.376754    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-data" (OuterVolumeSpecName: "istio-data") pod "4f9585cf-0d91-4424-946f-baf80cd5ca35" (UID: "4f9585cf-0d91-4424-946f-baf80cd5ca35"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:00:33.376929 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.376782    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-credential-socket\") pod \"4f9585cf-0d91-4424-946f-baf80cd5ca35\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") "
Sep 15 12:00:33.376929 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.376813    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f9585cf-0d91-4424-946f-baf80cd5ca35-istiod-ca-cert\") pod \"4f9585cf-0d91-4424-946f-baf80cd5ca35\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") "
Sep 15 12:00:33.376929 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.376845    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-workload-socket\") pod \"4f9585cf-0d91-4424-946f-baf80cd5ca35\" (UID: \"4f9585cf-0d91-4424-946f-baf80cd5ca35\") "
Sep 15 12:00:33.377159 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.376955    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4f9585cf-0d91-4424-946f-baf80cd5ca35" (UID: "4f9585cf-0d91-4424-946f-baf80cd5ca35"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:00:33.377221 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.377180    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-data\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:33.377221 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.377202    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-workload-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:33.377336 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.377257    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f9585cf-0d91-4424-946f-baf80cd5ca35-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4f9585cf-0d91-4424-946f-baf80cd5ca35" (UID: "4f9585cf-0d91-4424-946f-baf80cd5ca35"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:00:33.377455 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.377430    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4f9585cf-0d91-4424-946f-baf80cd5ca35" (UID: "4f9585cf-0d91-4424-946f-baf80cd5ca35"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:00:33.377546 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.377524    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4f9585cf-0d91-4424-946f-baf80cd5ca35" (UID: "4f9585cf-0d91-4424-946f-baf80cd5ca35"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:00:33.379049 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.379025    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f9585cf-0d91-4424-946f-baf80cd5ca35-kube-api-access-q7csr" (OuterVolumeSpecName: "kube-api-access-q7csr") pod "4f9585cf-0d91-4424-946f-baf80cd5ca35" (UID: "4f9585cf-0d91-4424-946f-baf80cd5ca35"). InnerVolumeSpecName "kube-api-access-q7csr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:00:33.379182 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.379065    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4f9585cf-0d91-4424-946f-baf80cd5ca35" (UID: "4f9585cf-0d91-4424-946f-baf80cd5ca35"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:00:33.379255 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.379242    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4f9585cf-0d91-4424-946f-baf80cd5ca35" (UID: "4f9585cf-0d91-4424-946f-baf80cd5ca35"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 12:00:33.379513 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.379494    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-token" (OuterVolumeSpecName: "istio-token") pod "4f9585cf-0d91-4424-946f-baf80cd5ca35" (UID: "4f9585cf-0d91-4424-946f-baf80cd5ca35"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:00:33.478369 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.478325    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-envoy\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:33.478369 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.478355    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-token\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:33.478369 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.478366    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f9585cf-0d91-4424-946f-baf80cd5ca35-istio-podinfo\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:33.478369 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.478374    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-credential-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:33.478369 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.478382    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f9585cf-0d91-4424-946f-baf80cd5ca35-istiod-ca-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:33.478857 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.478390    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f9585cf-0d91-4424-946f-baf80cd5ca35-workload-socket\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:33.478857 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:33.478400    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7csr\" (UniqueName: \"kubernetes.io/projected/4f9585cf-0d91-4424-946f-baf80cd5ca35-kube-api-access-q7csr\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:00:34.053094 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:34.053057    2560 generic.go:356] "Generic (PLEG): container finished" podID="4f9585cf-0d91-4424-946f-baf80cd5ca35" containerID="7e00fe2567e160ec018bebae171ff207b363804f2692279f9dc5f3ba9e9d974d" exitCode=0
Sep 15 12:00:34.053478 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:34.053101    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"
Sep 15 12:00:34.053478 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:34.053145    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj" event={"ID":"4f9585cf-0d91-4424-946f-baf80cd5ca35","Type":"ContainerDied","Data":"7e00fe2567e160ec018bebae171ff207b363804f2692279f9dc5f3ba9e9d974d"}
Sep 15 12:00:34.053478 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:34.053197    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj" event={"ID":"4f9585cf-0d91-4424-946f-baf80cd5ca35","Type":"ContainerDied","Data":"618deed1dceae75a925718ef2182fe976b29c690b39abeec50d06bb8157a71dd"}
Sep 15 12:00:34.053478 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:34.053218    2560 scope.go:117] "RemoveContainer" containerID="7e00fe2567e160ec018bebae171ff207b363804f2692279f9dc5f3ba9e9d974d"
Sep 15 12:00:34.062372 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:34.062355    2560 scope.go:117] "RemoveContainer" containerID="1eafb4e205e7cc771ee92a6155c40fc546435862ac89d90dd927bacedaf35828"
Sep 15 12:00:34.070679 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:34.070663    2560 scope.go:117] "RemoveContainer" containerID="7e00fe2567e160ec018bebae171ff207b363804f2692279f9dc5f3ba9e9d974d"
Sep 15 12:00:34.070944 ip-10-0-138-153 kubenswrapper[2560]: E0915 12:00:34.070922    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e00fe2567e160ec018bebae171ff207b363804f2692279f9dc5f3ba9e9d974d\": container with ID starting with 7e00fe2567e160ec018bebae171ff207b363804f2692279f9dc5f3ba9e9d974d not found: ID does not exist" containerID="7e00fe2567e160ec018bebae171ff207b363804f2692279f9dc5f3ba9e9d974d"
Sep 15 12:00:34.071043 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:34.070950    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e00fe2567e160ec018bebae171ff207b363804f2692279f9dc5f3ba9e9d974d"} err="failed to get container status \"7e00fe2567e160ec018bebae171ff207b363804f2692279f9dc5f3ba9e9d974d\": rpc error: code = NotFound desc = could not find container \"7e00fe2567e160ec018bebae171ff207b363804f2692279f9dc5f3ba9e9d974d\": container with ID starting with 7e00fe2567e160ec018bebae171ff207b363804f2692279f9dc5f3ba9e9d974d not found: ID does not exist"
Sep 15 12:00:34.071043 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:34.070967    2560 scope.go:117] "RemoveContainer" containerID="1eafb4e205e7cc771ee92a6155c40fc546435862ac89d90dd927bacedaf35828"
Sep 15 12:00:34.071200 ip-10-0-138-153 kubenswrapper[2560]: E0915 12:00:34.071186    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eafb4e205e7cc771ee92a6155c40fc546435862ac89d90dd927bacedaf35828\": container with ID starting with 1eafb4e205e7cc771ee92a6155c40fc546435862ac89d90dd927bacedaf35828 not found: ID does not exist" containerID="1eafb4e205e7cc771ee92a6155c40fc546435862ac89d90dd927bacedaf35828"
Sep 15 12:00:34.071245 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:34.071203    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eafb4e205e7cc771ee92a6155c40fc546435862ac89d90dd927bacedaf35828"} err="failed to get container status \"1eafb4e205e7cc771ee92a6155c40fc546435862ac89d90dd927bacedaf35828\": rpc error: code = NotFound desc = could not find container \"1eafb4e205e7cc771ee92a6155c40fc546435862ac89d90dd927bacedaf35828\": container with ID starting with 1eafb4e205e7cc771ee92a6155c40fc546435862ac89d90dd927bacedaf35828 not found: ID does not exist"
Sep 15 12:00:34.075017 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:34.074993    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"]
Sep 15 12:00:34.078719 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:34.078683    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2cc790-openshift-default-6746f98cd6-4xqjj"]
Sep 15 12:00:35.034446 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:00:35.034413    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f9585cf-0d91-4424-946f-baf80cd5ca35" path="/var/lib/kubelet/pods/4f9585cf-0d91-4424-946f-baf80cd5ca35/volumes"
Sep 15 12:01:32.125775 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.125739    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c8fb6dfc5-blqbp"]
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126107    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="550f07c6-e31d-4f18-afed-369d6684700a" containerName="istio-proxy"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126119    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="550f07c6-e31d-4f18-afed-369d6684700a" containerName="istio-proxy"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126128    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12ca1834-cc47-44d1-9af2-91d5d92e4cc4" containerName="payload-processing"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126134    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="12ca1834-cc47-44d1-9af2-91d5d92e4cc4" containerName="payload-processing"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126143    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f9585cf-0d91-4424-946f-baf80cd5ca35" containerName="istio-proxy"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126148    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f9585cf-0d91-4424-946f-baf80cd5ca35" containerName="istio-proxy"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126157    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f9585cf-0d91-4424-946f-baf80cd5ca35" containerName="istio-proxy"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126163    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f9585cf-0d91-4424-946f-baf80cd5ca35" containerName="istio-proxy"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126172    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerName="istio-proxy"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126177    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerName="istio-proxy"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126184    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bda31857-2d1b-42f5-b873-c6e37f9ce119" containerName="cleanup"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126189    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bda31857-2d1b-42f5-b873-c6e37f9ce119" containerName="cleanup"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126207    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerName="istio-proxy"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126215    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerName="istio-proxy"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126223    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerName="istio-proxy"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126228    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerName="istio-proxy"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126236    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerName="istio-proxy"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126242    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="edee2ae4-9151-4b76-9e8d-78cc89fe70ba" containerName="istio-proxy"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126251    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a050db2-5725-4e16-97b0-1da533d92eee" containerName="manager"
Sep 15 12:01:32.126272 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126256    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a050db2-5725-4e16-97b0-1da533d92eee" containerName="manager"
Sep 15 12:01:32.127051 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126333    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerName="istio-proxy"
Sep 15 12:01:32.127051 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126343    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e31b7b9-6f7b-403f-b310-551df113f4bb" containerName="istio-proxy"
Sep 15 12:01:32.127051 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126352    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f9585cf-0d91-4424-946f-baf80cd5ca35" containerName="istio-proxy"
Sep 15 12:01:32.127051 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126358    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="071de3b0-e4d5-43a0-b167-1356abd89f1b" containerName="istio-proxy"
Sep 15 12:01:32.127051 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126363    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bda31857-2d1b-42f5-b873-c6e37f9ce119" containerName="cleanup"
Sep 15 12:01:32.127051 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126368    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a050db2-5725-4e16-97b0-1da533d92eee" containerName="manager"
Sep 15 12:01:32.127051 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126373    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="12ca1834-cc47-44d1-9af2-91d5d92e4cc4" containerName="payload-processing"
Sep 15 12:01:32.127051 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.126379    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f9585cf-0d91-4424-946f-baf80cd5ca35" containerName="istio-proxy"
Sep 15 12:01:32.129743 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.129726    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp"
Sep 15 12:01:32.132771 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.132726    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 15 12:01:32.132771 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.132730    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-969bz\""
Sep 15 12:01:32.132771 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.132731    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 15 12:01:32.132978 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.132914    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 15 12:01:32.139957 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.139932    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c8fb6dfc5-blqbp"]
Sep 15 12:01:32.180484 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.180452    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpkz8\" (UniqueName: \"kubernetes.io/projected/b5301085-dc69-4f42-8b50-c3bbc3167ae3-kube-api-access-cpkz8\") pod \"maas-controller-5c8fb6dfc5-blqbp\" (UID: \"b5301085-dc69-4f42-8b50-c3bbc3167ae3\") " pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp"
Sep 15 12:01:32.180656 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.180514    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5301085-dc69-4f42-8b50-c3bbc3167ae3-metrics-certs\") pod \"maas-controller-5c8fb6dfc5-blqbp\" (UID: \"b5301085-dc69-4f42-8b50-c3bbc3167ae3\") " pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp"
Sep 15 12:01:32.180656 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.180573    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5301085-dc69-4f42-8b50-c3bbc3167ae3-webhook-cert\") pod \"maas-controller-5c8fb6dfc5-blqbp\" (UID: \"b5301085-dc69-4f42-8b50-c3bbc3167ae3\") " pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp"
Sep 15 12:01:32.281685 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.281648    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpkz8\" (UniqueName: \"kubernetes.io/projected/b5301085-dc69-4f42-8b50-c3bbc3167ae3-kube-api-access-cpkz8\") pod \"maas-controller-5c8fb6dfc5-blqbp\" (UID: \"b5301085-dc69-4f42-8b50-c3bbc3167ae3\") " pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp"
Sep 15 12:01:32.281847 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.281718    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5301085-dc69-4f42-8b50-c3bbc3167ae3-metrics-certs\") pod \"maas-controller-5c8fb6dfc5-blqbp\" (UID: \"b5301085-dc69-4f42-8b50-c3bbc3167ae3\") " pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp"
Sep 15 12:01:32.281847 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.281758    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5301085-dc69-4f42-8b50-c3bbc3167ae3-webhook-cert\") pod \"maas-controller-5c8fb6dfc5-blqbp\" (UID: \"b5301085-dc69-4f42-8b50-c3bbc3167ae3\") " pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp"
Sep 15 12:01:32.284208 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.284181    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5301085-dc69-4f42-8b50-c3bbc3167ae3-webhook-cert\") pod \"maas-controller-5c8fb6dfc5-blqbp\" (UID: \"b5301085-dc69-4f42-8b50-c3bbc3167ae3\") " pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp"
Sep 15 12:01:32.284326 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.284206    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5301085-dc69-4f42-8b50-c3bbc3167ae3-metrics-certs\") pod \"maas-controller-5c8fb6dfc5-blqbp\" (UID: \"b5301085-dc69-4f42-8b50-c3bbc3167ae3\") " pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp"
Sep 15 12:01:32.289315 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.289294    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpkz8\" (UniqueName: \"kubernetes.io/projected/b5301085-dc69-4f42-8b50-c3bbc3167ae3-kube-api-access-cpkz8\") pod \"maas-controller-5c8fb6dfc5-blqbp\" (UID: \"b5301085-dc69-4f42-8b50-c3bbc3167ae3\") " pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp"
Sep 15 12:01:32.441222 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.441151    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp"
Sep 15 12:01:32.713556 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.713530    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c8fb6dfc5-blqbp"]
Sep 15 12:01:32.715800 ip-10-0-138-153 kubenswrapper[2560]: W0915 12:01:32.715772    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5301085_dc69_4f42_8b50_c3bbc3167ae3.slice/crio-034e9959a673c19c2abf86d3a0a28e14ffb82c02d8685d38cbaa318134ed45aa WatchSource:0}: Error finding container 034e9959a673c19c2abf86d3a0a28e14ffb82c02d8685d38cbaa318134ed45aa: Status 404 returned error can't find the container with id 034e9959a673c19c2abf86d3a0a28e14ffb82c02d8685d38cbaa318134ed45aa
Sep 15 12:01:32.717145 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:32.717125    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 12:01:33.341097 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:33.341056    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp" event={"ID":"b5301085-dc69-4f42-8b50-c3bbc3167ae3","Type":"ContainerStarted","Data":"c54be75bdd8fba24ef676590cfd5937e6ce19ccd9e70665e47327e38841c3a4f"}
Sep 15 12:01:33.341488 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:33.341110    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp" event={"ID":"b5301085-dc69-4f42-8b50-c3bbc3167ae3","Type":"ContainerStarted","Data":"034e9959a673c19c2abf86d3a0a28e14ffb82c02d8685d38cbaa318134ed45aa"}
Sep 15 12:01:33.341488 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:33.341252    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp"
Sep 15 12:01:33.361456 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:33.361413    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp" podStartSLOduration=0.817590917 podStartE2EDuration="1.361400882s" totalImagesPullingTime="543.809965ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:01:33.360743077 +0000 UTC m=+2194.853520225" watchObservedRunningTime="2026-09-15 12:01:33.361400882 +0000 UTC m=+2194.854178029"
Sep 15 12:01:44.348018 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:44.347981    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp"
Sep 15 12:01:53.541112 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:53.541077    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:01:53.633637 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:53.633593    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c8fb6dfc5-blqbp"]
Sep 15 12:01:53.633913 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:53.633852    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp" podUID="b5301085-dc69-4f42-8b50-c3bbc3167ae3" containerName="manager" containerID="cri-o://c54be75bdd8fba24ef676590cfd5937e6ce19ccd9e70665e47327e38841c3a4f" gracePeriod=30
Sep 15 12:01:53.888462 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:53.888429    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp"
Sep 15 12:01:53.975421 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:53.975388    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5301085-dc69-4f42-8b50-c3bbc3167ae3-metrics-certs\") pod \"b5301085-dc69-4f42-8b50-c3bbc3167ae3\" (UID: \"b5301085-dc69-4f42-8b50-c3bbc3167ae3\") "
Sep 15 12:01:53.975421 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:53.975426    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5301085-dc69-4f42-8b50-c3bbc3167ae3-webhook-cert\") pod \"b5301085-dc69-4f42-8b50-c3bbc3167ae3\" (UID: \"b5301085-dc69-4f42-8b50-c3bbc3167ae3\") "
Sep 15 12:01:53.975693 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:53.975554    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpkz8\" (UniqueName: \"kubernetes.io/projected/b5301085-dc69-4f42-8b50-c3bbc3167ae3-kube-api-access-cpkz8\") pod \"b5301085-dc69-4f42-8b50-c3bbc3167ae3\" (UID: \"b5301085-dc69-4f42-8b50-c3bbc3167ae3\") "
Sep 15 12:01:53.977617 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:53.977590    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5301085-dc69-4f42-8b50-c3bbc3167ae3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b5301085-dc69-4f42-8b50-c3bbc3167ae3" (UID: "b5301085-dc69-4f42-8b50-c3bbc3167ae3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:01:53.977783 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:53.977640    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5301085-dc69-4f42-8b50-c3bbc3167ae3-kube-api-access-cpkz8" (OuterVolumeSpecName: "kube-api-access-cpkz8") pod "b5301085-dc69-4f42-8b50-c3bbc3167ae3" (UID: "b5301085-dc69-4f42-8b50-c3bbc3167ae3"). InnerVolumeSpecName "kube-api-access-cpkz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:01:53.977783 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:53.977671    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5301085-dc69-4f42-8b50-c3bbc3167ae3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b5301085-dc69-4f42-8b50-c3bbc3167ae3" (UID: "b5301085-dc69-4f42-8b50-c3bbc3167ae3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:01:54.076977 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:54.076901    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5301085-dc69-4f42-8b50-c3bbc3167ae3-metrics-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:01:54.076977 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:54.076926    2560 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5301085-dc69-4f42-8b50-c3bbc3167ae3-webhook-cert\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:01:54.076977 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:54.076935    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpkz8\" (UniqueName: \"kubernetes.io/projected/b5301085-dc69-4f42-8b50-c3bbc3167ae3-kube-api-access-cpkz8\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:01:54.444446 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:54.444406    2560 generic.go:356] "Generic (PLEG): container finished" podID="b5301085-dc69-4f42-8b50-c3bbc3167ae3" containerID="c54be75bdd8fba24ef676590cfd5937e6ce19ccd9e70665e47327e38841c3a4f" exitCode=0
Sep 15 12:01:54.444731 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:54.444450    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp"
Sep 15 12:01:54.444731 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:54.444482    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp" event={"ID":"b5301085-dc69-4f42-8b50-c3bbc3167ae3","Type":"ContainerDied","Data":"c54be75bdd8fba24ef676590cfd5937e6ce19ccd9e70665e47327e38841c3a4f"}
Sep 15 12:01:54.444731 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:54.444544    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c8fb6dfc5-blqbp" event={"ID":"b5301085-dc69-4f42-8b50-c3bbc3167ae3","Type":"ContainerDied","Data":"034e9959a673c19c2abf86d3a0a28e14ffb82c02d8685d38cbaa318134ed45aa"}
Sep 15 12:01:54.444731 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:54.444595    2560 scope.go:117] "RemoveContainer" containerID="c54be75bdd8fba24ef676590cfd5937e6ce19ccd9e70665e47327e38841c3a4f"
Sep 15 12:01:54.454480 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:54.454459    2560 scope.go:117] "RemoveContainer" containerID="c54be75bdd8fba24ef676590cfd5937e6ce19ccd9e70665e47327e38841c3a4f"
Sep 15 12:01:54.454775 ip-10-0-138-153 kubenswrapper[2560]: E0915 12:01:54.454755    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c54be75bdd8fba24ef676590cfd5937e6ce19ccd9e70665e47327e38841c3a4f\": container with ID starting with c54be75bdd8fba24ef676590cfd5937e6ce19ccd9e70665e47327e38841c3a4f not found: ID does not exist" containerID="c54be75bdd8fba24ef676590cfd5937e6ce19ccd9e70665e47327e38841c3a4f"
Sep 15 12:01:54.454853 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:54.454788    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c54be75bdd8fba24ef676590cfd5937e6ce19ccd9e70665e47327e38841c3a4f"} err="failed to get container status \"c54be75bdd8fba24ef676590cfd5937e6ce19ccd9e70665e47327e38841c3a4f\": rpc error: code = NotFound desc = could not find container \"c54be75bdd8fba24ef676590cfd5937e6ce19ccd9e70665e47327e38841c3a4f\": container with ID starting with c54be75bdd8fba24ef676590cfd5937e6ce19ccd9e70665e47327e38841c3a4f not found: ID does not exist"
Sep 15 12:01:54.468554 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:54.468532    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c8fb6dfc5-blqbp"]
Sep 15 12:01:54.471998 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:54.471977    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5c8fb6dfc5-blqbp"]
Sep 15 12:01:55.034239 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.034207    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5301085-dc69-4f42-8b50-c3bbc3167ae3" path="/var/lib/kubelet/pods/b5301085-dc69-4f42-8b50-c3bbc3167ae3/volumes"
Sep 15 12:01:55.110356 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.110303    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:01:55.221110 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.221079    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c8fb6dfc5-2nqr6"]
Sep 15 12:01:55.221455 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.221440    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5301085-dc69-4f42-8b50-c3bbc3167ae3" containerName="manager"
Sep 15 12:01:55.221582 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.221458    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5301085-dc69-4f42-8b50-c3bbc3167ae3" containerName="manager"
Sep 15 12:01:55.221582 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.221533    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5301085-dc69-4f42-8b50-c3bbc3167ae3" containerName="manager"
Sep 15 12:01:55.226050 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.226030    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6"
Sep 15 12:01:55.229333 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.229312    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-969bz\""
Sep 15 12:01:55.229437 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.229334    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 15 12:01:55.229437 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.229317    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 15 12:01:55.229437 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.229312    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 15 12:01:55.237726 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.237707    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c8fb6dfc5-2nqr6"]
Sep 15 12:01:55.389129 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.389096    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea07a716-52d7-48e9-8e10-c75263fd0d58-metrics-certs\") pod \"maas-controller-5c8fb6dfc5-2nqr6\" (UID: \"ea07a716-52d7-48e9-8e10-c75263fd0d58\") " pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6"
Sep 15 12:01:55.389308 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.389161    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ea07a716-52d7-48e9-8e10-c75263fd0d58-webhook-cert\") pod \"maas-controller-5c8fb6dfc5-2nqr6\" (UID: \"ea07a716-52d7-48e9-8e10-c75263fd0d58\") " pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6"
Sep 15 12:01:55.389308 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.389204    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf8bs\" (UniqueName: \"kubernetes.io/projected/ea07a716-52d7-48e9-8e10-c75263fd0d58-kube-api-access-hf8bs\") pod \"maas-controller-5c8fb6dfc5-2nqr6\" (UID: \"ea07a716-52d7-48e9-8e10-c75263fd0d58\") " pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6"
Sep 15 12:01:55.490253 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.490221    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ea07a716-52d7-48e9-8e10-c75263fd0d58-webhook-cert\") pod \"maas-controller-5c8fb6dfc5-2nqr6\" (UID: \"ea07a716-52d7-48e9-8e10-c75263fd0d58\") " pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6"
Sep 15 12:01:55.490414 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.490267    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf8bs\" (UniqueName: \"kubernetes.io/projected/ea07a716-52d7-48e9-8e10-c75263fd0d58-kube-api-access-hf8bs\") pod \"maas-controller-5c8fb6dfc5-2nqr6\" (UID: \"ea07a716-52d7-48e9-8e10-c75263fd0d58\") " pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6"
Sep 15 12:01:55.490414 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.490332    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea07a716-52d7-48e9-8e10-c75263fd0d58-metrics-certs\") pod \"maas-controller-5c8fb6dfc5-2nqr6\" (UID: \"ea07a716-52d7-48e9-8e10-c75263fd0d58\") " pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6"
Sep 15 12:01:55.492846 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.492822    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ea07a716-52d7-48e9-8e10-c75263fd0d58-webhook-cert\") pod \"maas-controller-5c8fb6dfc5-2nqr6\" (UID: \"ea07a716-52d7-48e9-8e10-c75263fd0d58\") " pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6"
Sep 15 12:01:55.492968 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.492878    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea07a716-52d7-48e9-8e10-c75263fd0d58-metrics-certs\") pod \"maas-controller-5c8fb6dfc5-2nqr6\" (UID: \"ea07a716-52d7-48e9-8e10-c75263fd0d58\") " pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6"
Sep 15 12:01:55.498657 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.498638    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf8bs\" (UniqueName: \"kubernetes.io/projected/ea07a716-52d7-48e9-8e10-c75263fd0d58-kube-api-access-hf8bs\") pod \"maas-controller-5c8fb6dfc5-2nqr6\" (UID: \"ea07a716-52d7-48e9-8e10-c75263fd0d58\") " pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6"
Sep 15 12:01:55.536612 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.536580    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6"
Sep 15 12:01:55.739993 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.739904    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:01:55.807413 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:55.807376    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c8fb6dfc5-2nqr6"]
Sep 15 12:01:55.811117 ip-10-0-138-153 kubenswrapper[2560]: W0915 12:01:55.811085    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea07a716_52d7_48e9_8e10_c75263fd0d58.slice/crio-56ccf48979142e684c060948540f79ac0a7b7efab1d6c7f122474f4d861b2ef8 WatchSource:0}: Error finding container 56ccf48979142e684c060948540f79ac0a7b7efab1d6c7f122474f4d861b2ef8: Status 404 returned error can't find the container with id 56ccf48979142e684c060948540f79ac0a7b7efab1d6c7f122474f4d861b2ef8
Sep 15 12:01:56.457292 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:56.457189    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6" event={"ID":"ea07a716-52d7-48e9-8e10-c75263fd0d58","Type":"ContainerStarted","Data":"625102812bef40cd7815a6fa3d9120d9b0ae677bf2b1892f9f71c69abb666d52"}
Sep 15 12:01:56.457292 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:56.457244    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6" event={"ID":"ea07a716-52d7-48e9-8e10-c75263fd0d58","Type":"ContainerStarted","Data":"56ccf48979142e684c060948540f79ac0a7b7efab1d6c7f122474f4d861b2ef8"}
Sep 15 12:01:56.457292 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:56.457276    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6"
Sep 15 12:01:56.476593 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:01:56.476531    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6" podStartSLOduration=1.157734716 podStartE2EDuration="1.476517789s" totalImagesPullingTime="318.783073ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:01:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:01:56.47523579 +0000 UTC m=+2217.968012938" watchObservedRunningTime="2026-09-15 12:01:56.476517789 +0000 UTC m=+2217.969294944"
Sep 15 12:02:07.464514 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:02:07.464477    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c8fb6dfc5-2nqr6"
Sep 15 12:02:49.466954 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:02:49.466918    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:02:49.536240 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:02:49.536207    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:02:56.396857 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:02:56.396820    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:02:56.497147 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:02:56.497116    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:03:02.478943 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:02.478895    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"]
Sep 15 12:03:02.480989 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:02.479155    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr" podUID="540f3743-3391-43a0-a98a-178d5e4cee78" containerName="manager" containerID="cri-o://6207f467fa4e2836c40b53fd5f834ef02ac8ab8aecb9145c52910e99930e65eb" gracePeriod=10
Sep 15 12:03:04.425983 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.425958    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"
Sep 15 12:03:04.504245 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.504213    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzw26\" (UniqueName: \"kubernetes.io/projected/540f3743-3391-43a0-a98a-178d5e4cee78-kube-api-access-hzw26\") pod \"540f3743-3391-43a0-a98a-178d5e4cee78\" (UID: \"540f3743-3391-43a0-a98a-178d5e4cee78\") "
Sep 15 12:03:04.504416 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.504281    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/540f3743-3391-43a0-a98a-178d5e4cee78-extensions-socket-volume\") pod \"540f3743-3391-43a0-a98a-178d5e4cee78\" (UID: \"540f3743-3391-43a0-a98a-178d5e4cee78\") "
Sep 15 12:03:04.504734 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.504698    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/540f3743-3391-43a0-a98a-178d5e4cee78-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "540f3743-3391-43a0-a98a-178d5e4cee78" (UID: "540f3743-3391-43a0-a98a-178d5e4cee78"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 12:03:04.506352 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.506324    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/540f3743-3391-43a0-a98a-178d5e4cee78-kube-api-access-hzw26" (OuterVolumeSpecName: "kube-api-access-hzw26") pod "540f3743-3391-43a0-a98a-178d5e4cee78" (UID: "540f3743-3391-43a0-a98a-178d5e4cee78"). InnerVolumeSpecName "kube-api-access-hzw26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:03:04.605357 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.605324    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/540f3743-3391-43a0-a98a-178d5e4cee78-extensions-socket-volume\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:03:04.605357 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.605351    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzw26\" (UniqueName: \"kubernetes.io/projected/540f3743-3391-43a0-a98a-178d5e4cee78-kube-api-access-hzw26\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:03:04.808441 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.808403    2560 generic.go:356] "Generic (PLEG): container finished" podID="540f3743-3391-43a0-a98a-178d5e4cee78" containerID="6207f467fa4e2836c40b53fd5f834ef02ac8ab8aecb9145c52910e99930e65eb" exitCode=0
Sep 15 12:03:04.808658 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.808453    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"
Sep 15 12:03:04.808658 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.808489    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr" event={"ID":"540f3743-3391-43a0-a98a-178d5e4cee78","Type":"ContainerDied","Data":"6207f467fa4e2836c40b53fd5f834ef02ac8ab8aecb9145c52910e99930e65eb"}
Sep 15 12:03:04.808658 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.808553    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr" event={"ID":"540f3743-3391-43a0-a98a-178d5e4cee78","Type":"ContainerDied","Data":"b580f6e7cd61080f3d3f42667e617f2367cddca3259502d9d675c55c5403a2ce"}
Sep 15 12:03:04.808658 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.808627    2560 scope.go:117] "RemoveContainer" containerID="6207f467fa4e2836c40b53fd5f834ef02ac8ab8aecb9145c52910e99930e65eb"
Sep 15 12:03:04.817906 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.817888    2560 scope.go:117] "RemoveContainer" containerID="519f7a86b0ec28b7e14b53c584cdfe61c4b8584ce32f7ce5b555063d6d30ec82"
Sep 15 12:03:04.828974 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.828954    2560 scope.go:117] "RemoveContainer" containerID="6207f467fa4e2836c40b53fd5f834ef02ac8ab8aecb9145c52910e99930e65eb"
Sep 15 12:03:04.829470 ip-10-0-138-153 kubenswrapper[2560]: E0915 12:03:04.829309    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6207f467fa4e2836c40b53fd5f834ef02ac8ab8aecb9145c52910e99930e65eb\": container with ID starting with 6207f467fa4e2836c40b53fd5f834ef02ac8ab8aecb9145c52910e99930e65eb not found: ID does not exist" containerID="6207f467fa4e2836c40b53fd5f834ef02ac8ab8aecb9145c52910e99930e65eb"
Sep 15 12:03:04.829470 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.829348    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6207f467fa4e2836c40b53fd5f834ef02ac8ab8aecb9145c52910e99930e65eb"} err="failed to get container status \"6207f467fa4e2836c40b53fd5f834ef02ac8ab8aecb9145c52910e99930e65eb\": rpc error: code = NotFound desc = could not find container \"6207f467fa4e2836c40b53fd5f834ef02ac8ab8aecb9145c52910e99930e65eb\": container with ID starting with 6207f467fa4e2836c40b53fd5f834ef02ac8ab8aecb9145c52910e99930e65eb not found: ID does not exist"
Sep 15 12:03:04.829470 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.829371    2560 scope.go:117] "RemoveContainer" containerID="519f7a86b0ec28b7e14b53c584cdfe61c4b8584ce32f7ce5b555063d6d30ec82"
Sep 15 12:03:04.829704 ip-10-0-138-153 kubenswrapper[2560]: E0915 12:03:04.829662    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"519f7a86b0ec28b7e14b53c584cdfe61c4b8584ce32f7ce5b555063d6d30ec82\": container with ID starting with 519f7a86b0ec28b7e14b53c584cdfe61c4b8584ce32f7ce5b555063d6d30ec82 not found: ID does not exist" containerID="519f7a86b0ec28b7e14b53c584cdfe61c4b8584ce32f7ce5b555063d6d30ec82"
Sep 15 12:03:04.829757 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.829712    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"519f7a86b0ec28b7e14b53c584cdfe61c4b8584ce32f7ce5b555063d6d30ec82"} err="failed to get container status \"519f7a86b0ec28b7e14b53c584cdfe61c4b8584ce32f7ce5b555063d6d30ec82\": rpc error: code = NotFound desc = could not find container \"519f7a86b0ec28b7e14b53c584cdfe61c4b8584ce32f7ce5b555063d6d30ec82\": container with ID starting with 519f7a86b0ec28b7e14b53c584cdfe61c4b8584ce32f7ce5b555063d6d30ec82 not found: ID does not exist"
Sep 15 12:03:04.830386 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.830368    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"]
Sep 15 12:03:04.835340 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:04.835318    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-m4jzr"]
Sep 15 12:03:05.033818 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:05.033722    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="540f3743-3391-43a0-a98a-178d5e4cee78" path="/var/lib/kubelet/pods/540f3743-3391-43a0-a98a-178d5e4cee78/volumes"
Sep 15 12:03:40.489047 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:40.489000    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:03:40.591376 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:40.591335    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:03:56.915776 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:56.915689    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:03:57.016105 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:03:57.016069    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:04:03.173820 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.173787    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9"]
Sep 15 12:04:03.174200 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.174149    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="540f3743-3391-43a0-a98a-178d5e4cee78" containerName="manager"
Sep 15 12:04:03.174200 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.174162    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="540f3743-3391-43a0-a98a-178d5e4cee78" containerName="manager"
Sep 15 12:04:03.174200 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.174180    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="540f3743-3391-43a0-a98a-178d5e4cee78" containerName="manager"
Sep 15 12:04:03.174200 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.174185    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="540f3743-3391-43a0-a98a-178d5e4cee78" containerName="manager"
Sep 15 12:04:03.174337 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.174240    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="540f3743-3391-43a0-a98a-178d5e4cee78" containerName="manager"
Sep 15 12:04:03.174337 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.174249    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="540f3743-3391-43a0-a98a-178d5e4cee78" containerName="manager"
Sep 15 12:04:03.178611 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.178597    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9"
Sep 15 12:04:03.180607 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.180585    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qslhn\""
Sep 15 12:04:03.190919 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.190899    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9"]
Sep 15 12:04:03.312126 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.312090    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/691195f3-c8ae-4be0-8fcf-85bf218a5e9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-jt7p9\" (UID: \"691195f3-c8ae-4be0-8fcf-85bf218a5e9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9"
Sep 15 12:04:03.312288 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.312151    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trvlz\" (UniqueName: \"kubernetes.io/projected/691195f3-c8ae-4be0-8fcf-85bf218a5e9f-kube-api-access-trvlz\") pod \"kuadrant-operator-controller-manager-5769f99d9f-jt7p9\" (UID: \"691195f3-c8ae-4be0-8fcf-85bf218a5e9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9"
Sep 15 12:04:03.413000 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.412964    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/691195f3-c8ae-4be0-8fcf-85bf218a5e9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-jt7p9\" (UID: \"691195f3-c8ae-4be0-8fcf-85bf218a5e9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9"
Sep 15 12:04:03.413137 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.413043    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trvlz\" (UniqueName: \"kubernetes.io/projected/691195f3-c8ae-4be0-8fcf-85bf218a5e9f-kube-api-access-trvlz\") pod \"kuadrant-operator-controller-manager-5769f99d9f-jt7p9\" (UID: \"691195f3-c8ae-4be0-8fcf-85bf218a5e9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9"
Sep 15 12:04:03.413321 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.413299    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/691195f3-c8ae-4be0-8fcf-85bf218a5e9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-jt7p9\" (UID: \"691195f3-c8ae-4be0-8fcf-85bf218a5e9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9"
Sep 15 12:04:03.428064 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.428011    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trvlz\" (UniqueName: \"kubernetes.io/projected/691195f3-c8ae-4be0-8fcf-85bf218a5e9f-kube-api-access-trvlz\") pod \"kuadrant-operator-controller-manager-5769f99d9f-jt7p9\" (UID: \"691195f3-c8ae-4be0-8fcf-85bf218a5e9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9"
Sep 15 12:04:03.489058 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.489037    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9"
Sep 15 12:04:03.777798 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:03.777770    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9"]
Sep 15 12:04:03.779456 ip-10-0-138-153 kubenswrapper[2560]: W0915 12:04:03.779432    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod691195f3_c8ae_4be0_8fcf_85bf218a5e9f.slice/crio-0706fb158306d96bda6aba3c3d15a98b940226ad1cee619d64326426bca5eebd WatchSource:0}: Error finding container 0706fb158306d96bda6aba3c3d15a98b940226ad1cee619d64326426bca5eebd: Status 404 returned error can't find the container with id 0706fb158306d96bda6aba3c3d15a98b940226ad1cee619d64326426bca5eebd
Sep 15 12:04:04.105289 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:04.105254    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9" event={"ID":"691195f3-c8ae-4be0-8fcf-85bf218a5e9f","Type":"ContainerStarted","Data":"4b6b890524e3ca51ee885967e2b1b97f07a1150ae4f9ed4aef119ca4c898fe28"}
Sep 15 12:04:04.105462 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:04.105313    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9"
Sep 15 12:04:04.105462 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:04.105330    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9" event={"ID":"691195f3-c8ae-4be0-8fcf-85bf218a5e9f","Type":"ContainerStarted","Data":"0706fb158306d96bda6aba3c3d15a98b940226ad1cee619d64326426bca5eebd"}
Sep 15 12:04:04.125224 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:04.125168    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9" podStartSLOduration=1.125153307 podStartE2EDuration="1.125153307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:04:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:04:04.123229219 +0000 UTC m=+2345.616006368" watchObservedRunningTime="2026-09-15 12:04:04.125153307 +0000 UTC m=+2345.617930454"
Sep 15 12:04:15.111432 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:15.111395    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-jt7p9"
Sep 15 12:04:59.739370 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:59.739341    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 12:04:59.746869 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:04:59.746845    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 12:05:00.586760 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:05:00.586721    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:05:16.479578 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:05:16.479536    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:05:57.435857 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:05:57.435824    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:05:58.395212 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:05:58.395170    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:06:04.170720 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:04.170682    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:06:17.483810 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.483775    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"]
Sep 15 12:06:17.487813 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.487787    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:17.494598 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.494550    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"]
Sep 15 12:06:17.594352 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.594320    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2-maas-api-tls\") pod \"maas-api-64765b4c74-89xtf\" (UID: \"30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2\") " pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:17.594572 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.594379    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2-metrics-certs\") pod \"maas-api-64765b4c74-89xtf\" (UID: \"30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2\") " pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:17.594572 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.594450    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2-openshift-service-ca\") pod \"maas-api-64765b4c74-89xtf\" (UID: \"30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2\") " pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:17.594572 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.594490    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x49fr\" (UniqueName: \"kubernetes.io/projected/30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2-kube-api-access-x49fr\") pod \"maas-api-64765b4c74-89xtf\" (UID: \"30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2\") " pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:17.695235 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.695195    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2-openshift-service-ca\") pod \"maas-api-64765b4c74-89xtf\" (UID: \"30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2\") " pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:17.695235 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.695243    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x49fr\" (UniqueName: \"kubernetes.io/projected/30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2-kube-api-access-x49fr\") pod \"maas-api-64765b4c74-89xtf\" (UID: \"30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2\") " pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:17.695460 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.695308    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2-maas-api-tls\") pod \"maas-api-64765b4c74-89xtf\" (UID: \"30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2\") " pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:17.695460 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.695351    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2-metrics-certs\") pod \"maas-api-64765b4c74-89xtf\" (UID: \"30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2\") " pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:17.696063 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.696034    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2-openshift-service-ca\") pod \"maas-api-64765b4c74-89xtf\" (UID: \"30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2\") " pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:17.697889 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.697869    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2-metrics-certs\") pod \"maas-api-64765b4c74-89xtf\" (UID: \"30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2\") " pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:17.697993 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.697912    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2-maas-api-tls\") pod \"maas-api-64765b4c74-89xtf\" (UID: \"30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2\") " pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:17.702748 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.702725    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x49fr\" (UniqueName: \"kubernetes.io/projected/30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2-kube-api-access-x49fr\") pod \"maas-api-64765b4c74-89xtf\" (UID: \"30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2\") " pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:17.803774 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:17.803395    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:18.096758 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:18.096722    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"]
Sep 15 12:06:18.101761 ip-10-0-138-153 kubenswrapper[2560]: W0915 12:06:18.101712    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30e5485d_5cd9_4a62_bb6d_1ed2ed9c16e2.slice/crio-770f82c24c55698082129826369d200390de9ff456a3fd4c238eea3732df4a1b WatchSource:0}: Error finding container 770f82c24c55698082129826369d200390de9ff456a3fd4c238eea3732df4a1b: Status 404 returned error can't find the container with id 770f82c24c55698082129826369d200390de9ff456a3fd4c238eea3732df4a1b
Sep 15 12:06:18.789430 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:18.789391    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf" event={"ID":"30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2","Type":"ContainerStarted","Data":"eae525ab6ae1b9493b85a4fcee91e1c401c80d11a2c7dc0a4a70fff77f43a046"}
Sep 15 12:06:18.789861 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:18.789449    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf" event={"ID":"30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2","Type":"ContainerStarted","Data":"770f82c24c55698082129826369d200390de9ff456a3fd4c238eea3732df4a1b"}
Sep 15 12:06:18.789861 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:18.789497    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:18.808830 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:18.808783    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf" podStartSLOduration=1.3043078430000001 podStartE2EDuration="1.808769291s" totalImagesPullingTime="504.461448ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 12:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:06:18.806530931 +0000 UTC m=+2480.299308073" watchObservedRunningTime="2026-09-15 12:06:18.808769291 +0000 UTC m=+2480.301546438"
Sep 15 12:06:24.800886 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:24.800857    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-64765b4c74-89xtf"
Sep 15 12:06:34.525983 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:34.525941    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"]
Sep 15 12:06:34.526427 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:34.526220    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq" podUID="df018f63-5cc1-4614-9fae-ca23d05bd862" containerName="maas-api" containerID="cri-o://12c6532a2b7ac2495a76f87225015649484705c886a6f165ba1fb267d4a0643a" gracePeriod=30
Sep 15 12:06:35.778042 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.778010    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 12:06:35.862855 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.862823    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqkzj\" (UniqueName: \"kubernetes.io/projected/df018f63-5cc1-4614-9fae-ca23d05bd862-kube-api-access-rqkzj\") pod \"df018f63-5cc1-4614-9fae-ca23d05bd862\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") "
Sep 15 12:06:35.863031 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.862884    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df018f63-5cc1-4614-9fae-ca23d05bd862-maas-api-tls\") pod \"df018f63-5cc1-4614-9fae-ca23d05bd862\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") "
Sep 15 12:06:35.863031 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.862910    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df018f63-5cc1-4614-9fae-ca23d05bd862-metrics-certs\") pod \"df018f63-5cc1-4614-9fae-ca23d05bd862\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") "
Sep 15 12:06:35.863031 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.862953    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df018f63-5cc1-4614-9fae-ca23d05bd862-openshift-service-ca\") pod \"df018f63-5cc1-4614-9fae-ca23d05bd862\" (UID: \"df018f63-5cc1-4614-9fae-ca23d05bd862\") "
Sep 15 12:06:35.863363 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.863336    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df018f63-5cc1-4614-9fae-ca23d05bd862-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "df018f63-5cc1-4614-9fae-ca23d05bd862" (UID: "df018f63-5cc1-4614-9fae-ca23d05bd862"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 12:06:35.865025 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.864993    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df018f63-5cc1-4614-9fae-ca23d05bd862-kube-api-access-rqkzj" (OuterVolumeSpecName: "kube-api-access-rqkzj") pod "df018f63-5cc1-4614-9fae-ca23d05bd862" (UID: "df018f63-5cc1-4614-9fae-ca23d05bd862"). InnerVolumeSpecName "kube-api-access-rqkzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 12:06:35.865131 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.865021    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df018f63-5cc1-4614-9fae-ca23d05bd862-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "df018f63-5cc1-4614-9fae-ca23d05bd862" (UID: "df018f63-5cc1-4614-9fae-ca23d05bd862"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:06:35.865131 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.865059    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df018f63-5cc1-4614-9fae-ca23d05bd862-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "df018f63-5cc1-4614-9fae-ca23d05bd862" (UID: "df018f63-5cc1-4614-9fae-ca23d05bd862"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 12:06:35.875685 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.875659    2560 generic.go:356] "Generic (PLEG): container finished" podID="df018f63-5cc1-4614-9fae-ca23d05bd862" containerID="12c6532a2b7ac2495a76f87225015649484705c886a6f165ba1fb267d4a0643a" exitCode=0
Sep 15 12:06:35.875788 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.875703    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"
Sep 15 12:06:35.875788 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.875749    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq" event={"ID":"df018f63-5cc1-4614-9fae-ca23d05bd862","Type":"ContainerDied","Data":"12c6532a2b7ac2495a76f87225015649484705c886a6f165ba1fb267d4a0643a"}
Sep 15 12:06:35.875868 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.875808    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq" event={"ID":"df018f63-5cc1-4614-9fae-ca23d05bd862","Type":"ContainerDied","Data":"aa92729cd3e394b54f5ac2e0d4ea297dfa3608f54f615a366bf73b4881dfc211"}
Sep 15 12:06:35.875868 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.875840    2560 scope.go:117] "RemoveContainer" containerID="12c6532a2b7ac2495a76f87225015649484705c886a6f165ba1fb267d4a0643a"
Sep 15 12:06:35.886888 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.886871    2560 scope.go:117] "RemoveContainer" containerID="12c6532a2b7ac2495a76f87225015649484705c886a6f165ba1fb267d4a0643a"
Sep 15 12:06:35.887195 ip-10-0-138-153 kubenswrapper[2560]: E0915 12:06:35.887175    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12c6532a2b7ac2495a76f87225015649484705c886a6f165ba1fb267d4a0643a\": container with ID starting with 12c6532a2b7ac2495a76f87225015649484705c886a6f165ba1fb267d4a0643a not found: ID does not exist" containerID="12c6532a2b7ac2495a76f87225015649484705c886a6f165ba1fb267d4a0643a"
Sep 15 12:06:35.887304 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.887221    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12c6532a2b7ac2495a76f87225015649484705c886a6f165ba1fb267d4a0643a"} err="failed to get container status \"12c6532a2b7ac2495a76f87225015649484705c886a6f165ba1fb267d4a0643a\": rpc error: code = NotFound desc = could not find container \"12c6532a2b7ac2495a76f87225015649484705c886a6f165ba1fb267d4a0643a\": container with ID starting with 12c6532a2b7ac2495a76f87225015649484705c886a6f165ba1fb267d4a0643a not found: ID does not exist"
Sep 15 12:06:35.896490 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.896466    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"]
Sep 15 12:06:35.902022 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.901998    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-57d8d6c56b-t9gmq"]
Sep 15 12:06:35.963546 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.963522    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqkzj\" (UniqueName: \"kubernetes.io/projected/df018f63-5cc1-4614-9fae-ca23d05bd862-kube-api-access-rqkzj\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:06:35.963546 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.963543    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df018f63-5cc1-4614-9fae-ca23d05bd862-maas-api-tls\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:06:35.963703 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.963553    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df018f63-5cc1-4614-9fae-ca23d05bd862-metrics-certs\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:06:35.963703 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:35.963583    2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df018f63-5cc1-4614-9fae-ca23d05bd862-openshift-service-ca\") on node \"ip-10-0-138-153.ec2.internal\" DevicePath \"\""
Sep 15 12:06:37.037463 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:37.037416    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df018f63-5cc1-4614-9fae-ca23d05bd862" path="/var/lib/kubelet/pods/df018f63-5cc1-4614-9fae-ca23d05bd862/volumes"
Sep 15 12:06:56.599473 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:56.599389    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-kd2kc_dd5db2e0-f577-40eb-8ec9-7af6169ef256/manager/0.log"
Sep 15 12:06:56.721524 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:56.721494    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c8fb6dfc5-2nqr6_ea07a716-52d7-48e9-8e10-c75263fd0d58/manager/0.log"
Sep 15 12:06:57.294212 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:57.294184    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b96c7c888-n2l58_a9f74621-3355-4e42-a550-d7500eeec2d1/copy-manifests/0.log"
Sep 15 12:06:57.301101 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:57.301071    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b96c7c888-n2l58_a9f74621-3355-4e42-a550-d7500eeec2d1/manager/0.log"
Sep 15 12:06:57.726362 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:57.726330    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-64765b4c74-89xtf_30e5485d-5cd9-4a62-bb6d-1ed2ed9c16e2/maas-api/0.log"
Sep 15 12:06:57.837897 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:57.837863    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29824545-jjjxz_68f3ab4b-a83f-43e2-b96e-d90b3dab9a3d/cleanup/0.log"
Sep 15 12:06:58.048199 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:58.048118    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-5st7g_e0334b89-a740-419c-83b2-e55f5bcd9855/postgres/0.log"
Sep 15 12:06:58.908485 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:58.908451    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-lg4gt_bba73fe3-5ba7-4c50-bb0e-7e5efe7b2c5a/manager/0.log"
Sep 15 12:06:59.023918 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:59.023888    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-lf6rx_dc1335fa-fa3b-4fce-9e0f-4bb61c0dabe7/manager/0.log"
Sep 15 12:06:59.249729 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:59.249661    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-jt7p9_691195f3-c8ae-4be0-8fcf-85bf218a5e9f/manager/0.log"
Sep 15 12:06:59.930791 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:06:59.930757    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pjn7w_37b6d3ea-fd8f-4958-8246-4ef0aced2bb8/discovery/0.log"
Sep 15 12:07:00.032445 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:00.032417    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79674675df-hkgzr_feaf279e-bf6c-4cfd-9778-fccc641b7150/kube-auth-proxy/0.log"
Sep 15 12:07:00.049619 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:00.049586    2560 scope.go:117] "RemoveContainer" containerID="90ac9785fd7f935e5ccf6f3f64540004e66f62f4326bc454517a6a9d49491fa8"
Sep 15 12:07:00.248207 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:00.248125    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-pgr4n_f4e91b9c-b31e-48a0-bb33-c4cd19625eea/istio-proxy/1.log"
Sep 15 12:07:00.368494 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:00.368461    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-f9vl9_abe51807-1597-4321-8154-f4e185610305/payload-pre-processing/0.log"
Sep 15 12:07:00.584435 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:00.584357    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568d798994-hm7cl_6cdd8d68-d249-46a5-9eed-37d9cb155861/router/0.log"
Sep 15 12:07:01.012127 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:01.012094    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-mcxtm_d29b9b99-a2d6-4d10-acab-e488bc14b721/main/0.log"
Sep 15 12:07:01.326910 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:01.326811    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq6cl5_c23523bc-df65-48a2-9e95-60caab38b465/main/0.log"
Sep 15 12:07:06.967337 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:06.967299    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:07:12.970551 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:12.970519    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4jcwq_7f0c4ecd-c730-4a1e-ac07-fdecb4781a4b/global-pull-secret-syncer/0.log"
Sep 15 12:07:13.101888 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:13.101857    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log"
Sep 15 12:07:13.233695 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:13.233594    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-153.ec2.internal_1142610769d3e76a0d4f81b8e01ff4c0/haproxy/0.log"
Sep 15 12:07:14.638015 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:14.637978    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:07:18.027138 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:18.027051    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-lg4gt_bba73fe3-5ba7-4c50-bb0e-7e5efe7b2c5a/manager/0.log"
Sep 15 12:07:18.052024 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:18.052001    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-lf6rx_dc1335fa-fa3b-4fce-9e0f-4bb61c0dabe7/manager/0.log"
Sep 15 12:07:18.115457 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:18.115429    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-jt7p9_691195f3-c8ae-4be0-8fcf-85bf218a5e9f/manager/0.log"
Sep 15 12:07:20.131771 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:20.131732    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-7q7fj_34006888-4d27-47db-a314-26878e17bdff/cluster-monitoring-operator/0.log"
Sep 15 12:07:20.368154 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:20.368122    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gffmw_776c84f2-fb70-455a-aa6b-ea7cbdbd8889/node-exporter/0.log"
Sep 15 12:07:20.388694 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:20.388626    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gffmw_776c84f2-fb70-455a-aa6b-ea7cbdbd8889/kube-rbac-proxy/0.log"
Sep 15 12:07:20.407685 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:20.407666    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gffmw_776c84f2-fb70-455a-aa6b-ea7cbdbd8889/init-textfile/0.log"
Sep 15 12:07:20.617128 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:20.617103    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2778a33d-a7b1-4dc9-82bc-31058249360d/prometheus/0.log"
Sep 15 12:07:20.636769 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:20.636746    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2778a33d-a7b1-4dc9-82bc-31058249360d/config-reloader/0.log"
Sep 15 12:07:20.656700 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:20.656633    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2778a33d-a7b1-4dc9-82bc-31058249360d/thanos-sidecar/0.log"
Sep 15 12:07:20.676242 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:20.676223    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2778a33d-a7b1-4dc9-82bc-31058249360d/kube-rbac-proxy-web/0.log"
Sep 15 12:07:20.704242 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:20.704220    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2778a33d-a7b1-4dc9-82bc-31058249360d/kube-rbac-proxy/0.log"
Sep 15 12:07:20.723254 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:20.723233    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2778a33d-a7b1-4dc9-82bc-31058249360d/kube-rbac-proxy-thanos/0.log"
Sep 15 12:07:20.749503 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:20.749473    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2778a33d-a7b1-4dc9-82bc-31058249360d/init-config-reloader/0.log"
Sep 15 12:07:23.203872 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.203847    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/1.log"
Sep 15 12:07:23.212112 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.212082    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dw8hm_46058d0a-ac82-4cd3-9482-faf534e13869/console-operator/2.log"
Sep 15 12:07:23.728115 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.728080    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"]
Sep 15 12:07:23.728460 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.728447    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df018f63-5cc1-4614-9fae-ca23d05bd862" containerName="maas-api"
Sep 15 12:07:23.728513 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.728461    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="df018f63-5cc1-4614-9fae-ca23d05bd862" containerName="maas-api"
Sep 15 12:07:23.728578 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.728528    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="df018f63-5cc1-4614-9fae-ca23d05bd862" containerName="maas-api"
Sep 15 12:07:23.731904 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.731884    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:23.733763 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.733743    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nkxzt\"/\"default-dockercfg-jpntf\""
Sep 15 12:07:23.733888 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.733747    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nkxzt\"/\"openshift-service-ca.crt\""
Sep 15 12:07:23.734367 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.734346    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nkxzt\"/\"kube-root-ca.crt\""
Sep 15 12:07:23.743092 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.743067    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"]
Sep 15 12:07:23.868189 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.868163    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-wbz22_e96ecc2f-c511-4b3b-a0cf-888686ce3940/download-server/0.log"
Sep 15 12:07:23.882723 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.882698    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0d527a50-4532-4415-a298-1b6f1fb8d9ea-podres\") pod \"perf-node-gather-daemonset-kkm7d\" (UID: \"0d527a50-4532-4415-a298-1b6f1fb8d9ea\") " pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:23.882828 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.882735    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0d527a50-4532-4415-a298-1b6f1fb8d9ea-proc\") pod \"perf-node-gather-daemonset-kkm7d\" (UID: \"0d527a50-4532-4415-a298-1b6f1fb8d9ea\") " pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:23.882828 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.882766    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0d527a50-4532-4415-a298-1b6f1fb8d9ea-sys\") pod \"perf-node-gather-daemonset-kkm7d\" (UID: \"0d527a50-4532-4415-a298-1b6f1fb8d9ea\") " pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:23.882899 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.882835    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0d527a50-4532-4415-a298-1b6f1fb8d9ea-lib-modules\") pod \"perf-node-gather-daemonset-kkm7d\" (UID: \"0d527a50-4532-4415-a298-1b6f1fb8d9ea\") " pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:23.882899 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.882881    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb5tf\" (UniqueName: \"kubernetes.io/projected/0d527a50-4532-4415-a298-1b6f1fb8d9ea-kube-api-access-xb5tf\") pod \"perf-node-gather-daemonset-kkm7d\" (UID: \"0d527a50-4532-4415-a298-1b6f1fb8d9ea\") " pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:23.983387 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.983318    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0d527a50-4532-4415-a298-1b6f1fb8d9ea-sys\") pod \"perf-node-gather-daemonset-kkm7d\" (UID: \"0d527a50-4532-4415-a298-1b6f1fb8d9ea\") " pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:23.983387 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.983358    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0d527a50-4532-4415-a298-1b6f1fb8d9ea-lib-modules\") pod \"perf-node-gather-daemonset-kkm7d\" (UID: \"0d527a50-4532-4415-a298-1b6f1fb8d9ea\") " pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:23.983535 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.983432    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0d527a50-4532-4415-a298-1b6f1fb8d9ea-sys\") pod \"perf-node-gather-daemonset-kkm7d\" (UID: \"0d527a50-4532-4415-a298-1b6f1fb8d9ea\") " pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:23.983535 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.983461    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb5tf\" (UniqueName: \"kubernetes.io/projected/0d527a50-4532-4415-a298-1b6f1fb8d9ea-kube-api-access-xb5tf\") pod \"perf-node-gather-daemonset-kkm7d\" (UID: \"0d527a50-4532-4415-a298-1b6f1fb8d9ea\") " pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:23.983535 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.983510    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0d527a50-4532-4415-a298-1b6f1fb8d9ea-podres\") pod \"perf-node-gather-daemonset-kkm7d\" (UID: \"0d527a50-4532-4415-a298-1b6f1fb8d9ea\") " pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:23.983663 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.983554    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0d527a50-4532-4415-a298-1b6f1fb8d9ea-lib-modules\") pod \"perf-node-gather-daemonset-kkm7d\" (UID: \"0d527a50-4532-4415-a298-1b6f1fb8d9ea\") " pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:23.983663 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.983627    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0d527a50-4532-4415-a298-1b6f1fb8d9ea-podres\") pod \"perf-node-gather-daemonset-kkm7d\" (UID: \"0d527a50-4532-4415-a298-1b6f1fb8d9ea\") " pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:23.983663 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.983625    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0d527a50-4532-4415-a298-1b6f1fb8d9ea-proc\") pod \"perf-node-gather-daemonset-kkm7d\" (UID: \"0d527a50-4532-4415-a298-1b6f1fb8d9ea\") " pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:23.983762 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.983672    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0d527a50-4532-4415-a298-1b6f1fb8d9ea-proc\") pod \"perf-node-gather-daemonset-kkm7d\" (UID: \"0d527a50-4532-4415-a298-1b6f1fb8d9ea\") " pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:23.991649 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:23.991621    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb5tf\" (UniqueName: \"kubernetes.io/projected/0d527a50-4532-4415-a298-1b6f1fb8d9ea-kube-api-access-xb5tf\") pod \"perf-node-gather-daemonset-kkm7d\" (UID: \"0d527a50-4532-4415-a298-1b6f1fb8d9ea\") " pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:24.048376 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:24.048353    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:24.320771 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:24.320741    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"]
Sep 15 12:07:24.322546 ip-10-0-138-153 kubenswrapper[2560]: W0915 12:07:24.322517    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0d527a50_4532_4415_a298_1b6f1fb8d9ea.slice/crio-ea16e85648df180632423121c36ce8f27f1d40ddb7974054f0112c1accf0e0dc WatchSource:0}: Error finding container ea16e85648df180632423121c36ce8f27f1d40ddb7974054f0112c1accf0e0dc: Status 404 returned error can't find the container with id ea16e85648df180632423121c36ce8f27f1d40ddb7974054f0112c1accf0e0dc
Sep 15 12:07:24.594497 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:24.594468    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-6rnxm_a98bf294-4c61-4611-959d-e4941241a01b/volume-data-source-validator/0.log"
Sep 15 12:07:25.122716 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:25.122683    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d" event={"ID":"0d527a50-4532-4415-a298-1b6f1fb8d9ea","Type":"ContainerStarted","Data":"102cb5d90dff0bbf72f7016372d2cd868aea504c15d43a55edf8970c04730958"}
Sep 15 12:07:25.122884 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:25.122835    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:25.122949 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:25.122892    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d" event={"ID":"0d527a50-4532-4415-a298-1b6f1fb8d9ea","Type":"ContainerStarted","Data":"ea16e85648df180632423121c36ce8f27f1d40ddb7974054f0112c1accf0e0dc"}
Sep 15 12:07:25.138407 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:25.138354    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d" podStartSLOduration=2.138338245 podStartE2EDuration="2.138338245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 12:07:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 12:07:25.137808963 +0000 UTC m=+2546.630586101" watchObservedRunningTime="2026-09-15 12:07:25.138338245 +0000 UTC m=+2546.631115407"
Sep 15 12:07:25.818613 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:25.818581    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c4gb7_9c09db0b-a1bd-4fe6-9009-567294abff0f/dns/0.log"
Sep 15 12:07:25.838300 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:25.838278    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c4gb7_9c09db0b-a1bd-4fe6-9009-567294abff0f/kube-rbac-proxy/0.log"
Sep 15 12:07:25.953687 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:25.953667    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hq8r8_43830c40-9ac0-4143-b673-a73a6273075c/dns-node-resolver/0.log"
Sep 15 12:07:26.545068 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:26.545031    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xfzbq_dc0a2e63-3309-454b-890d-bc634e4ee43a/node-ca/0.log"
Sep 15 12:07:28.288651 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:28.288604    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pjn7w_37b6d3ea-fd8f-4958-8246-4ef0aced2bb8/discovery/0.log"
Sep 15 12:07:28.311551 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:28.311519    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79674675df-hkgzr_feaf279e-bf6c-4cfd-9778-fccc641b7150/kube-auth-proxy/0.log"
Sep 15 12:07:28.362096 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:28.362068    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-pgr4n_f4e91b9c-b31e-48a0-bb33-c4cd19625eea/istio-proxy/0.log"
Sep 15 12:07:28.366814 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:28.366794    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-pgr4n_f4e91b9c-b31e-48a0-bb33-c4cd19625eea/istio-proxy/1.log"
Sep 15 12:07:28.403008 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:28.402987    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-f9vl9_abe51807-1597-4321-8154-f4e185610305/payload-pre-processing/0.log"
Sep 15 12:07:28.462555 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:28.462534    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568d798994-hm7cl_6cdd8d68-d249-46a5-9eed-37d9cb155861/router/0.log"
Sep 15 12:07:29.386171 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:29.386143    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-prrz4_9067bd21-a937-4395-88e4-327269b84de3/serve-healthcheck-canary/0.log"
Sep 15 12:07:29.893738 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:29.893701    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-njsmc_2667a7e4-1546-43a1-b61d-01f8eefe3c82/insights-operator/0.log"
Sep 15 12:07:29.894664 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:29.894640    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-njsmc_2667a7e4-1546-43a1-b61d-01f8eefe3c82/insights-operator/1.log"
Sep 15 12:07:30.043256 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:30.043227    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-smdnj_61c49049-f9ab-48fd-8ee7-7b853de6abd1/kube-rbac-proxy/0.log"
Sep 15 12:07:30.064128 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:30.064087    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-smdnj_61c49049-f9ab-48fd-8ee7-7b853de6abd1/exporter/0.log"
Sep 15 12:07:30.085005 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:30.084979    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-smdnj_61c49049-f9ab-48fd-8ee7-7b853de6abd1/extractor/0.log"
Sep 15 12:07:30.520611 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:30.520556    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sb4n_02e080c4-4b79-42c3-9ad1-319b5ca2df4c/konnectivity-agent/0.log" limit=1024
Sep 15 12:07:31.136685 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:31.136651    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nkxzt/perf-node-gather-daemonset-kkm7d"
Sep 15 12:07:32.703066 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:32.702988    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-kd2kc_dd5db2e0-f577-40eb-8ec9-7af6169ef256/manager/0.log"
Sep 15 12:07:32.770328 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:32.770302    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c8fb6dfc5-2nqr6_ea07a716-52d7-48e9-8e10-c75263fd0d58/manager/0.log"
Sep 15 12:07:33.081701 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:33.081624    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b96c7c888-n2l58_a9f74621-3355-4e42-a550-d7500eeec2d1/manager/0.log"
Sep 15 12:07:33.104286 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:33.104263    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b96c7c888-n2l58_a9f74621-3355-4e42-a550-d7500eeec2d1/copy-manifests/0.log"
Sep 15 12:07:35.471656 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:35.471547    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pjn7w_37b6d3ea-fd8f-4958-8246-4ef0aced2bb8/discovery/0.log"
Sep 15 12:07:35.485806 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:35.485782    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79674675df-hkgzr_feaf279e-bf6c-4cfd-9778-fccc641b7150/kube-auth-proxy/0.log"
Sep 15 12:07:35.530766 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:35.530742    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-pgr4n_f4e91b9c-b31e-48a0-bb33-c4cd19625eea/istio-proxy/0.log"
Sep 15 12:07:35.535383 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:35.535356    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-pgr4n_f4e91b9c-b31e-48a0-bb33-c4cd19625eea/istio-proxy/1.log"
Sep 15 12:07:35.565195 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:35.565167    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-f9vl9_abe51807-1597-4321-8154-f4e185610305/payload-pre-processing/0.log"
Sep 15 12:07:35.609397 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:35.609368    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568d798994-hm7cl_6cdd8d68-d249-46a5-9eed-37d9cb155861/router/0.log"
Sep 15 12:07:37.935881 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:37.935849    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-pjn7w_37b6d3ea-fd8f-4958-8246-4ef0aced2bb8/discovery/0.log"
Sep 15 12:07:38.023908 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:38.023881    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79674675df-hkgzr_feaf279e-bf6c-4cfd-9778-fccc641b7150/kube-auth-proxy/0.log"
Sep 15 12:07:38.080132 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:38.080100    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-pgr4n_f4e91b9c-b31e-48a0-bb33-c4cd19625eea/istio-proxy/0.log"
Sep 15 12:07:38.085074 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:38.085050    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-pgr4n_f4e91b9c-b31e-48a0-bb33-c4cd19625eea/istio-proxy/1.log"
Sep 15 12:07:38.115049 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:38.115024    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-f9vl9_abe51807-1597-4321-8154-f4e185610305/payload-pre-processing/0.log"
Sep 15 12:07:38.167347 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:38.167325    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568d798994-hm7cl_6cdd8d68-d249-46a5-9eed-37d9cb155861/router/0.log"
Sep 15 12:07:40.535676 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:40.535628    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-7zt9z_21196aa4-a811-43f9-b006-1f448fce3ea7/kube-storage-version-migrator-operator/1.log"
Sep 15 12:07:40.537143 ip-10-0-138-153 kubenswrapper[2560]: I0915 12:07:40.537116    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-7zt9z_21196aa4-a811-43f9-b006-1f448fce3ea7/kube-storage-version-migrator-operator/0.log"