Sep 09 21:33:00.415242 ip-10-0-140-157 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 21:33:00.811240 ip-10-0-140-157 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 21:33:00.811240 ip-10-0-140-157 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 21:33:00.811240 ip-10-0-140-157 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 21:33:00.811240 ip-10-0-140-157 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 21:33:00.811240 ip-10-0-140-157 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 21:33:00.812270 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.812189    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 21:33:00.816015 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816000    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 21:33:00.816015 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816014    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816018    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816021    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816025    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816028    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816031    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816034    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816036    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816039    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816042    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816044    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816048    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816051    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816054    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816056    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816059    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816061    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816064    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816067    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 21:33:00.816078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816071    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816075    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816077    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816080    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816083    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816086    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816089    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816092    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816095    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816098    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816100    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816103    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816105    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816108    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816110    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816113    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816115    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816118    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816120    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816123    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 21:33:00.816532 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816126    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816129    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816132    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816134    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816137    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816139    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816142    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816144    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816149    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816151    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816154    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816157    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816160    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816162    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816166    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816169    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816171    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816174    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816177    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816179    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 21:33:00.817022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816182    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816185    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816187    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816190    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816192    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816195    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816197    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816200    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816202    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816205    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816208    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816210    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816213    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816215    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816217    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816220    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816222    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816225    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816229    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816233    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 21:33:00.817509 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816236    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816239    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816242    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816244    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816247    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816249    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816252    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816256    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816259    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816263    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816266    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816269    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816272    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816275    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816278    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816281    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816283    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816286    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 21:33:00.817998 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.816289    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 21:33:00.818533 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818521    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 21:33:00.818533 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818533    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 21:33:00.818596 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818536    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 21:33:00.818596 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818540    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 21:33:00.818596 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818543    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 21:33:00.818596 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818546    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 21:33:00.818596 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818549    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 21:33:00.818596 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818552    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 21:33:00.818596 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818573    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 21:33:00.818596 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818578    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 21:33:00.818596 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818581    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 21:33:00.818596 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818584    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 21:33:00.818596 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818587    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 21:33:00.818596 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818598    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 21:33:00.818596 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818601    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818604    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818607    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818610    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818613    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818617    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818621    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818624    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818627    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818630    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818633    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818636    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818639    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818641    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818644    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818647    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818649    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818652    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818654    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 21:33:00.818920 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818657    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818659    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818662    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818664    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818667    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818669    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818672    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818674    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818676    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818679    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818681    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818684    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818686    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818689    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818691    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818695    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818697    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818700    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818703    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818706    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 21:33:00.819397 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818709    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818711    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818714    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818716    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818718    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818721    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818723    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818726    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818729    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818731    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818733    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818736    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818738    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818740    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818743    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818745    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818748    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818752    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818756    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818759    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 21:33:00.819895 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818762    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818764    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818767    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818770    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818773    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818775    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818778    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818782    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818785    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818787    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818790    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818792    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818794    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818797    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818799    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818802    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818804    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818807    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818810    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 21:33:00.820390 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818813    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818816    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818819    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818822    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818825    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818828    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.818855    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819281    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819287    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819290    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819293    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819295    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819300    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819303    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819307    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819309    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819312    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819315    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819318    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 21:33:00.820850 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819321    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819324    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819326    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819329    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819331    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819334    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819337    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819339    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819342    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819344    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819347    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819349    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819351    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819354    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819357    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819359    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819361    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819364    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819366    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819376    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 21:33:00.821327 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819379    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819381    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819394    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819396    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819399    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819402    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819405    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819408    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819410    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819413    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819415    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819418    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819421    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819423    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819426    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819430    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819434    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819436    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819439    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819442    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 21:33:00.821836 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819444    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819447    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819450    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819454    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819457    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819460    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819463    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819466    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819469    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819472    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819475    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819478    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819486    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819489    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819492    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819494    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819497    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819500    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819503    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 21:33:00.822319 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819505    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819508    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819510    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819517    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819520    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819522    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819525    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819527    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819530    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819532    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819535    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819537    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819539    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819542    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819544    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819547    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819550    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819552    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819554    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819557    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 21:33:00.822814 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819559    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819562    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819565    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819567    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819571    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819573    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819581    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.819584    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819659    2578 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819699    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819710    2578 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819714    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819718    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819721    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819726    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819730    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819733    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819736    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819739    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819743    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819746    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819749    2578 flags.go:64] FLAG: --cgroup-root=""
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819752    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 21:33:00.823304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819754    2578 flags.go:64] FLAG: --client-ca-file=""
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819757    2578 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819760    2578 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819766    2578 flags.go:64] FLAG: --cluster-domain=""
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819769    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819772    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819775    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819778    2578 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819782    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819785    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819788    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819791    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819794    2578 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819797    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819800    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819804    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819812    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819816    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819819    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819822    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819825    2578 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819828    2578 flags.go:64] FLAG: --enable-server="true"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819831    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819838    2578 flags.go:64] FLAG: --event-burst="100"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819841    2578 flags.go:64] FLAG: --event-qps="50"
Sep 09 21:33:00.823851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819844    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819847    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819850    2578 flags.go:64] FLAG: --eviction-hard=""
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819854    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819857    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819860    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819862    2578 flags.go:64] FLAG: --eviction-soft=""
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819865    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819868    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819871    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819873    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819876    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819879    2578 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819882    2578 flags.go:64] FLAG: --feature-gates=""
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819885    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819888    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819891    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819894    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819897    2578 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819900    2578 flags.go:64] FLAG: --help="false"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819904    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-140-157.ec2.internal"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819907    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819910    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 21:33:00.824487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819913    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819922    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819925    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819928    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819931    2578 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819934    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819937    2578 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819939    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819943    2578 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819945    2578 flags.go:64] FLAG: --kube-reserved=""
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819948    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819951    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819954    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819956    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819959    2578 flags.go:64] FLAG: --lock-file=""
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819962    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819979    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819984    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819990    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819992    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.819995    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820001    2578 flags.go:64] FLAG: --logging-format="text"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820004    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820007    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 21:33:00.825083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820010    2578 flags.go:64] FLAG: --manifest-url=""
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820013    2578 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820017    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820020    2578 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820024    2578 flags.go:64] FLAG: --max-pods="110"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820027    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820030    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820033    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820036    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820039    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820050    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820053    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820060    2578 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820063    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820066    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820069    2578 flags.go:64] FLAG: --pod-cidr=""
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820072    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820077    2578 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820080    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820083    2578 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820085    2578 flags.go:64] FLAG: --port="10250"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820088    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820091    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-071086175cdbd410a"
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820094    2578 flags.go:64] FLAG: --qos-reserved=""
Sep 09 21:33:00.825660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820097    2578 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820100    2578 flags.go:64] FLAG: --register-node="true"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820103    2578 flags.go:64] FLAG: --register-with-taints=""
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820107    2578 flags.go:64] FLAG: --registry-burst="10"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820109    2578 flags.go:64] FLAG: --registry-qps="5"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820112    2578 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820116    2578 flags.go:64] FLAG: --reserved-memory=""
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820120    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820123    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820126    2578 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820129    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820131    2578 flags.go:64] FLAG: --runonce="false"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820134    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820138    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820141    2578 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820143    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820146    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820150    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820153    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820161    2578 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820164    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820167    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820170    2578 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820173    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820175    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 21:33:00.826265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820178    2578 flags.go:64] FLAG: --system-cgroups=""
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820181    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820186    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820189    2578 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820192    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820198    2578 flags.go:64] FLAG: --tls-min-version=""
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820201    2578 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820203    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820206    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820209    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820212    2578 flags.go:64] FLAG: --v="2"
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820215    2578 flags.go:64] FLAG: --version="false"
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820219    2578 flags.go:64] FLAG: --vmodule=""
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820224    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.820227    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820370    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820373    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820377    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820379    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820382    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820385    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820387    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 21:33:00.826869 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820390    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820393    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820396    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820398    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820401    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820411    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820414    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820417    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820419    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820422    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820424    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820427    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820429    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820432    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820434    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820437    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820439    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820442    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820444    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820446    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 21:33:00.827413 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820449    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820453    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820457    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820462    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820465    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820468    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820471    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820474    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820477    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820480    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820484    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820487    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820490    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820492    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820495    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820497    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820500    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820502    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820511    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 21:33:00.827934 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820514    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820516    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820519    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820521    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820524    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820526    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820528    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820531    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820533    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820536    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820538    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820541    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820543    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820546    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820550    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820552    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820556    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820559    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820561    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820564    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 21:33:00.828408 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820567    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820569    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820572    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820575    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820578    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820580    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820583    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820585    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820588    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820590    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820593    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820595    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820599    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820602    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820605    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820607    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820610    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820612    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820614    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820617    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 21:33:00.828905 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820619    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 21:33:00.829396 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820622    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 21:33:00.829396 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820624    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 21:33:00.829396 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820627    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 21:33:00.829396 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820629    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 21:33:00.829396 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820631    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 21:33:00.829396 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820634    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 21:33:00.829396 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820637    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 21:33:00.829396 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820640    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 21:33:00.829396 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820643    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 21:33:00.829396 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820645    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 21:33:00.829396 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820648    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 21:33:00.829396 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.820651    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 21:33:00.829396 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.821113    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 21:33:00.829396 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.828058    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 21:33:00.829396 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.828072    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.828822    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.828829    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828910    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828915    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828919    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828922    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828925    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828928    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828931    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828933    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828938    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828941    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828950    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828953    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828956    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828959    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828962    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828979    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 21:33:00.829765 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828983    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828986    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828989    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828993    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828996    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.828999    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829002    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829006    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829009    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829012    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829015    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829018    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829020    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829024    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829026    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829029    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829032    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829035    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829038    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 21:33:00.830238 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829040    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829043    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829045    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829047    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829050    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829053    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829055    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829058    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829066    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829069    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829071    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829073    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829076    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829078    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829081    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829084    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829086    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829088    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829091    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829093    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 21:33:00.830709 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829096    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829099    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829101    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829104    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829106    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829109    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829112    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829114    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829117    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829119    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829122    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829124    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829127    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829129    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829132    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829134    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829137    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829139    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829142    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829144    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 21:33:00.831241 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829147    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829154    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829157    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829165    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829168    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829170    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829173    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829176    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829178    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829181    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829183    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829186    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829189    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829191    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829194    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829196    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829200    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829203    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829206    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 21:33:00.831728 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829209    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 21:33:00.832224 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829211    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 21:33:00.832224 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829214    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 21:33:00.832224 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829217    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 21:33:00.832224 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829219    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 21:33:00.832224 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.829224    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 21:33:00.832224 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829368    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 21:33:00.832224 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829373    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 21:33:00.832224 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829376    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 21:33:00.832224 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829379    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 21:33:00.832224 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829382    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 21:33:00.832224 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829385    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 21:33:00.832224 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829387    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 21:33:00.832224 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829390    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 21:33:00.832224 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829392    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 21:33:00.832224 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829400    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829403    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829406    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829409    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829413    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829416    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829420    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829423    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829426    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829429    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829431    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829433    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829436    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829439    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829441    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829444    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829447    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829449    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829451    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 21:33:00.832601 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829454    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829457    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829459    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829462    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829464    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829467    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829471    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829474    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829477    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829480    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829483    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829486    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829488    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829491    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829498    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829501    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829504    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829506    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829509    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829511    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 21:33:00.833104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829514    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829517    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829519    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829521    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829524    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829526    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829529    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829531    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829534    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829537    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829539    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829542    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829544    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829546    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829549    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829551    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829554    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829556    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829558    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829561    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 21:33:00.833608 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829563    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829566    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829568    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829571    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829573    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829575    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829578    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829587    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829589    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829592    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829594    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829597    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829599    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829602    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829604    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829607    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829609    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829612    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829614    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829617    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 21:33:00.834318 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829619    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 21:33:00.835060 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829622    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 21:33:00.835060 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829624    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 21:33:00.835060 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829628    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 21:33:00.835060 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829631    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 21:33:00.835060 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829634    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 21:33:00.835060 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829636    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 21:33:00.835060 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829639    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 21:33:00.835060 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829642    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 21:33:00.835060 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829644    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 21:33:00.835060 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:00.829647    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 21:33:00.835060 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.829652    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 21:33:00.835060 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.829767    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 21:33:00.835060 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.832209    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 21:33:00.835060 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.833192    2578 server.go:1019] "Starting client certificate rotation"
Sep 09 21:33:00.835410 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.833294    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 21:33:00.835410 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.833340    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 21:33:00.837380 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.837364    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 21:33:00.838549 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.838532    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 21:33:00.848976 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.848951    2578 log.go:25] "Validated CRI v1 runtime API"
Sep 09 21:33:00.855083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.855067    2578 log.go:25] "Validated CRI v1 image API"
Sep 09 21:33:00.855896 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.855872    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 21:33:00.859771 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.859741    2578 fs.go:135] Filesystem UUIDs: map[1d34368d-98c0-4e16-a202-1cdd7bb84190:/dev/nvme0n1p4 57c466c8-c9ad-4eea-9ebd-534fa1fc40e6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 09 21:33:00.859849 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.859771    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 21:33:00.864303 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.864280    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 21:33:00.866101 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.865987    2578 manager.go:216] Machine: {Timestamp:2026-09-09 21:33:00.864782505 +0000 UTC m=+0.337747406 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099917 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e42f348796157bdc284f7362912f8 SystemUUID:ec2e42f3-4879-6157-bdc2-84f7362912f8 BootID:bada3e19-a055-4f48-80f8-f1901cef7c3d Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:91:b6:c1:a3:39 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:91:b6:c1:a3:39 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:21:75:6f:87:b0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 21:33:00.866101 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.866097    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 21:33:00.866273 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.866172    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 21:33:00.869461 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.869433    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 21:33:00.869602 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.869464    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-157.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 21:33:00.870175 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.870165    2578 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 21:33:00.870209 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.870178    2578 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 21:33:00.870209 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.870185    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 21:33:00.870886 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.870875    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 21:33:00.872012 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.872002    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 21:33:00.876150 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.876139    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 21:33:00.876269 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.876260    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 21:33:00.876324 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.876315    2578 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 21:33:00.876352 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.876332    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 21:33:00.876352 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.876345    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 21:33:00.876408 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.876353    2578 kubelet.go:398] "Adding apiserver pod source"
Sep 09 21:33:00.876408 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.876363    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 21:33:00.876815 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.876797    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 21:33:00.876875 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.876839    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 21:33:00.880557 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.879839    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 21:33:00.882531 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.882517    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 21:33:00.882627 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.882546    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 21:33:00.882712 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.882700    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 21:33:00.882744 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.882722    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 21:33:00.882744 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.882735    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 21:33:00.882744 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.882744    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 21:33:00.882827 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.882752    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 21:33:00.882827 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.882758    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 21:33:00.883205 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.883195    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 21:33:00.883205 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.883205    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 21:33:00.883268 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.883212    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 21:33:00.883268 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.883217    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 21:33:00.883268 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.883225    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 21:33:00.883268 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.883233    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 21:33:00.883880 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.883870    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 21:33:00.883912 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.883882    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 21:33:00.887902 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.887889    2578 server.go:1268] "Started kubelet"
Sep 09 21:33:00.888003 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.887955    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 21:33:00.888054 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.887991    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 21:33:00.888110 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.888065    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 21:33:00.888738 ip-10-0-140-157 systemd[1]: Started Kubernetes Kubelet.
Sep 09 21:33:00.888907 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:00.888855    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 21:33:00.888907 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.888864    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-157.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 21:33:00.889042 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:00.888918    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 21:33:00.889502 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.889478    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 21:33:00.890363 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.890350    2578 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 21:33:00.892475 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:00.892456    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 21:33:00.894573 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.894545    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 21:33:00.895228 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.895205    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 21:33:00.895428 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:00.895402    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-157.ec2.internal\" not found"
Sep 09 21:33:00.895824 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.895806    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 21:33:00.895918 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.895828    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 21:33:00.895918 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.895806    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 21:33:00.896038 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.895956    2578 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 21:33:00.896038 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.895963    2578 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 21:33:00.896185 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.896160    2578 factory.go:153] Registering CRI-O factory
Sep 09 21:33:00.896185 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.896179    2578 factory.go:223] Registration of the crio container factory successfully
Sep 09 21:33:00.896278 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.896226    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 21:33:00.896278 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.896234    2578 factory.go:55] Registering systemd factory
Sep 09 21:33:00.896278 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.896239    2578 factory.go:223] Registration of the systemd container factory successfully
Sep 09 21:33:00.896278 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.896252    2578 factory.go:103] Registering Raw factory
Sep 09 21:33:00.896278 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.896267    2578 manager.go:1196] Started watching for new ooms in manager
Sep 09 21:33:00.896671 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.896658    2578 manager.go:349] Starting recovery of all containers
Sep 09 21:33:00.897888 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:00.896958    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-157.ec2.internal.18d3c48b9bb22bf1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-157.ec2.internal,UID:ip-10-0-140-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-157.ec2.internal,},FirstTimestamp:2026-09-09 21:33:00.887862257 +0000 UTC m=+0.360827157,LastTimestamp:2026-09-09 21:33:00.887862257 +0000 UTC m=+0.360827157,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-157.ec2.internal,}"
Sep 09 21:33:00.898185 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:00.898165    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 09 21:33:00.901715 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:00.901693    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 09 21:33:00.908773 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.908560    2578 manager.go:354] Recovery completed
Sep 09 21:33:00.913855 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.913841    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 21:33:00.916595 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.916569    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 21:33:00.916662 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.916607    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 21:33:00.916662 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.916618    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasSufficientPID"
Sep 09 21:33:00.917195 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.917182    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 21:33:00.917261 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.917196    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 21:33:00.917261 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.917214    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 21:33:00.917261 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.917243    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 21:33:00.918050 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.918029    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 21:33:00.918144 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.918065    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 21:33:00.918144 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.918082    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasSufficientPID"
Sep 09 21:33:00.918467 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:00.918385    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-157.ec2.internal.18d3c48b9d689512  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-157.ec2.internal,UID:ip-10-0-140-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-157.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-157.ec2.internal,},FirstTimestamp:2026-09-09 21:33:00.916593938 +0000 UTC m=+0.389558842,LastTimestamp:2026-09-09 21:33:00.916593938 +0000 UTC m=+0.389558842,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-157.ec2.internal,}"
Sep 09 21:33:00.920089 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.920073    2578 policy_none.go:49] "None policy: Start"
Sep 09 21:33:00.920145 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.920093    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 21:33:00.920145 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.920107    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 21:33:00.921335 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.921323    2578 policy_none.go:47] "Start"
Sep 09 21:33:00.923090 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.923071    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qzd7n"
Sep 09 21:33:00.928488 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:00.928405    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-157.ec2.internal.18d3c48b9d68dcb7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-157.ec2.internal,UID:ip-10-0-140-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-157.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-157.ec2.internal,},FirstTimestamp:2026-09-09 21:33:00.916612279 +0000 UTC m=+0.389577183,LastTimestamp:2026-09-09 21:33:00.916612279 +0000 UTC m=+0.389577183,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-157.ec2.internal,}"
Sep 09 21:33:00.928698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.928680    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qzd7n"
Sep 09 21:33:00.956665 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.956650    2578 manager.go:337] "Starting Device Plugin manager"
Sep 09 21:33:00.960186 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:00.956684    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 21:33:00.960186 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.956696    2578 server.go:87] "Starting device plugin registration server"
Sep 09 21:33:00.960186 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.957073    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 21:33:00.960186 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.957086    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 21:33:00.960186 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.957222    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 21:33:00.960186 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.957315    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 21:33:00.960186 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.957324    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 21:33:00.960186 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:00.957803    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 21:33:00.960186 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:00.957867    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-157.ec2.internal\" not found"
Sep 09 21:33:00.995077 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.995044    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 21:33:00.996165 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.996147    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 21:33:00.996233 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.996206    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 21:33:00.996233 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.996231    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 21:33:00.996327 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:00.996274    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 09 21:33:00.998009 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:00.997990    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 21:33:01.057789 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.057769    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 21:33:01.058657 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.058638    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 21:33:01.058736 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.058667    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 21:33:01.058736 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.058680    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasSufficientPID"
Sep 09 21:33:01.058736 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.058701    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.067847 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.067781    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.067847 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:01.067801    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-157.ec2.internal\": node \"ip-10-0-140-157.ec2.internal\" not found"
Sep 09 21:33:01.090799 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:01.090777    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-157.ec2.internal\" not found"
Sep 09 21:33:01.096912 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.096899    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-157.ec2.internal"]
Sep 09 21:33:01.096957 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.096952    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 21:33:01.098473 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.098452    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 21:33:01.098563 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.098480    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 21:33:01.098563 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.098490    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasSufficientPID"
Sep 09 21:33:01.099829 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.099817    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 21:33:01.099946 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.099933    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.100000 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.099960    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 21:33:01.100518 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.100504    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 21:33:01.100580 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.100524    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 21:33:01.100580 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.100535    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasSufficientPID"
Sep 09 21:33:01.100580 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.100544    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 21:33:01.100580 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.100569    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 21:33:01.100687 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.100585    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasSufficientPID"
Sep 09 21:33:01.101747 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.101733    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.101836 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.101756    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 21:33:01.102391 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.102375    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 21:33:01.102473 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.102396    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 21:33:01.102473 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.102410    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeHasSufficientPID"
Sep 09 21:33:01.125608 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:01.125587    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-157.ec2.internal\" not found" node="ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.128843 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:01.128826    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-157.ec2.internal\" not found" node="ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.191509 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:01.191489    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-157.ec2.internal\" not found"
Sep 09 21:33:01.197706 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.197684    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f62cbe264ea5b90cb1c40b6d5e6165e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal\" (UID: \"f62cbe264ea5b90cb1c40b6d5e6165e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.197762 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.197714    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f62cbe264ea5b90cb1c40b6d5e6165e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal\" (UID: \"f62cbe264ea5b90cb1c40b6d5e6165e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.197762 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.197731    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/58cbb1667332061624da09e390b4d934-config\") pod \"kube-apiserver-proxy-ip-10-0-140-157.ec2.internal\" (UID: \"58cbb1667332061624da09e390b4d934\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.292293 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:01.292265    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-157.ec2.internal\" not found"
Sep 09 21:33:01.298645 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.298623    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f62cbe264ea5b90cb1c40b6d5e6165e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal\" (UID: \"f62cbe264ea5b90cb1c40b6d5e6165e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.298707 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.298650    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f62cbe264ea5b90cb1c40b6d5e6165e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal\" (UID: \"f62cbe264ea5b90cb1c40b6d5e6165e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.298707 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.298667    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/58cbb1667332061624da09e390b4d934-config\") pod \"kube-apiserver-proxy-ip-10-0-140-157.ec2.internal\" (UID: \"58cbb1667332061624da09e390b4d934\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.298707 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.298699    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/58cbb1667332061624da09e390b4d934-config\") pod \"kube-apiserver-proxy-ip-10-0-140-157.ec2.internal\" (UID: \"58cbb1667332061624da09e390b4d934\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.298806 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.298715    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f62cbe264ea5b90cb1c40b6d5e6165e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal\" (UID: \"f62cbe264ea5b90cb1c40b6d5e6165e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.298806 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.298731    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f62cbe264ea5b90cb1c40b6d5e6165e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal\" (UID: \"f62cbe264ea5b90cb1c40b6d5e6165e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.393096 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:01.393025    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-157.ec2.internal\" not found"
Sep 09 21:33:01.428472 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.428447    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.431358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.431339    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-157.ec2.internal"
Sep 09 21:33:01.493112 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:01.493081    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-157.ec2.internal\" not found"
Sep 09 21:33:01.593599 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:01.593576    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-157.ec2.internal\" not found"
Sep 09 21:33:01.694160 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:01.694103    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-157.ec2.internal\" not found"
Sep 09 21:33:01.796215 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:01.796184    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-157.ec2.internal\" not found"
Sep 09 21:33:01.835427 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.835397    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 21:33:01.835816 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.835560    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 21:33:01.896614 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:01.896587    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-157.ec2.internal\" not found"
Sep 09 21:33:01.930810 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.930775    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 21:28:01 +0000 UTC" deadline="2028-05-28 21:22:58.601367362 +0000 UTC"
Sep 09 21:33:01.930810 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:01.930808    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15047h49m56.670563834s"
Sep 09 21:33:01.998380 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:01.998319    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-157.ec2.internal\" not found"
Sep 09 21:33:02.076186 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.076168    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 21:33:02.096323 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.096303    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-157.ec2.internal"
Sep 09 21:33:02.107160 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.107144    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 21:33:02.108909 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.108895    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal"
Sep 09 21:33:02.123702 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.123563    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 21:33:02.173778 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:02.173753    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf62cbe264ea5b90cb1c40b6d5e6165e6.slice/crio-1be716b00c5256a9283b493ac1a9b0bfddf07d252af6fe2a9d5620720314daad WatchSource:0}: Error finding container 1be716b00c5256a9283b493ac1a9b0bfddf07d252af6fe2a9d5620720314daad: Status 404 returned error can't find the container with id 1be716b00c5256a9283b493ac1a9b0bfddf07d252af6fe2a9d5620720314daad
Sep 09 21:33:02.174127 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:02.174107    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58cbb1667332061624da09e390b4d934.slice/crio-ce1b32d305a7f02de8f74e237e6978c24f7a2de9cf4eab2790bc01ac17b18cdb WatchSource:0}: Error finding container ce1b32d305a7f02de8f74e237e6978c24f7a2de9cf4eab2790bc01ac17b18cdb: Status 404 returned error can't find the container with id ce1b32d305a7f02de8f74e237e6978c24f7a2de9cf4eab2790bc01ac17b18cdb
Sep 09 21:33:02.178755 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.178736    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 21:33:02.180810 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.180790    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 21:33:02.230920 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.230892    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 21:33:02.878734 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.878705    2578 apiserver.go:52] "Watching apiserver"
Sep 09 21:33:02.889856 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.889818    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 21:33:02.890325 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.890300    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal","openshift-multus/multus-4zq5s","openshift-multus/multus-additional-cni-plugins-rqzxb","openshift-network-diagnostics/network-check-target-bdlgh","kube-system/konnectivity-agent-m7d8c","kube-system/kube-apiserver-proxy-ip-10-0-140-157.ec2.internal","openshift-cluster-node-tuning-operator/tuned-qbfd2","openshift-dns/node-resolver-jrklh","openshift-image-registry/node-ca-wcng5","openshift-multus/network-metrics-daemon-sgvmj","openshift-network-operator/iptables-alerter-vdc7g","openshift-ovn-kubernetes/ovnkube-node-rftp9","kube-system/global-pull-secret-syncer-69ls5"]
Sep 09 21:33:02.892130 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.892096    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.893927 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.893905    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.894665 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.894645    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 21:33:02.894751 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.894703    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 21:33:02.895537 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.895131    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:02.895537 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.895449    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 21:33:02.895743 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.895660    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 21:33:02.895816 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.895777    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 21:33:02.896758 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.896165    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 21:33:02.896758 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.896455    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mxj4t\""
Sep 09 21:33:02.896758 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.896624    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 21:33:02.896975 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.896832    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 21:33:02.897033 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.897017    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 21:33:02.897250 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.897169    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 21:33:02.897328 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.897259    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 21:33:02.897523 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.897462    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qcxtt\""
Sep 09 21:33:02.897523 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.897520    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:02.897667 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:02.897594    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:02.898275 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.897866    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 21:33:02.898275 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.897906    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 21:33:02.898275 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.898122    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8cbxc\""
Sep 09 21:33:02.900856 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.900520    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m7d8c"
Sep 09 21:33:02.901641 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.901621    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jrklh"
Sep 09 21:33:02.902909 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.902889    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 21:33:02.903156 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.903139    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9zwlx\""
Sep 09 21:33:02.903244 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.903143    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 21:33:02.903373 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.903356    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:02.903985 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.903948    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dwrml\""
Sep 09 21:33:02.904642 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.904612    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 21:33:02.904720 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.904699    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 21:33:02.904921 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.904904    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:02.905065 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:02.905006    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:02.905520 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905424    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/414900e1-c678-4e61-9e76-0e2e29ac783c-cnibin\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:02.905520 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905459    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/414900e1-c678-4e61-9e76-0e2e29ac783c-cni-binary-copy\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:02.905520 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905487    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/414900e1-c678-4e61-9e76-0e2e29ac783c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:02.905723 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905524    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-etc-openvswitch\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.905723 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905570    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-kubelet\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.905723 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905600    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-run-netns\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.905723 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905624    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-run-systemd\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.905723 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905665    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-var-lib-openvswitch\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.905723 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905691    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-run-ovn-kubernetes\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.905723 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905695    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 21:33:02.905723 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905717    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-multus-cni-dir\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.906157 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905748    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-multus-socket-dir-parent\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.906157 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905801    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-var-lib-cni-multus\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.906157 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905863    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-var-lib-kubelet\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.906157 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905890    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cc8427f7-1d50-457a-879b-001799103caa-multus-daemon-config\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.906157 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905928    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/414900e1-c678-4e61-9e76-0e2e29ac783c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:02.906157 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905955    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc8427f7-1d50-457a-879b-001799103caa-cni-binary-copy\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.906157 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.905983    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxtfq\" (UniqueName: \"kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq\") pod \"network-check-target-bdlgh\" (UID: \"885c3bcb-44b1-46c4-ab1c-62269c8d5474\") " pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:02.906157 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906070    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-log-socket\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.906157 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906099    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-ovnkube-script-lib\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.906157 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906148    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-hostroot\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906177    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2v5q\" (UniqueName: \"kubernetes.io/projected/cc8427f7-1d50-457a-879b-001799103caa-kube-api-access-g2v5q\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906195    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906212    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-slash\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906238    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-cni-bin\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906278    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-ovnkube-config\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906340    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-run-k8s-cni-cncf-io\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906372    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-node-log\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906397    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-cni-netd\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906415    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906424    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-ovn-node-metrics-cert\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906450    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-system-cni-dir\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906526    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5f9ff\""
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906539    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-multus-conf-dir\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906566    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-run-multus-certs\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906623    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-etc-kubernetes\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.906652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906648    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/414900e1-c678-4e61-9e76-0e2e29ac783c-system-cni-dir\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:02.907452 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlsdv\" (UniqueName: \"kubernetes.io/projected/414900e1-c678-4e61-9e76-0e2e29ac783c-kube-api-access-nlsdv\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:02.907452 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906723    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-cnibin\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.907452 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906749    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-run-ovn\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.907452 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906769    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-systemd-units\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.907452 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906787    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-env-overrides\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.907452 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906841    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzf25\" (UniqueName: \"kubernetes.io/projected/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-kube-api-access-qzf25\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.907452 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906875    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-var-lib-cni-bin\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.907452 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906904    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/414900e1-c678-4e61-9e76-0e2e29ac783c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:02.907452 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906958    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-run-openvswitch\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.907452 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.906996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:02.907452 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.907018    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wcng5"
Sep 09 21:33:02.907452 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.907031    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-run-netns\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:02.908061 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.907607    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:02.908061 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:02.907689    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:02.909116 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.909031    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vdc7g"
Sep 09 21:33:02.909219 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.909158    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 21:33:02.909727 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.909354    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 21:33:02.909727 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.909501    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 21:33:02.909727 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.909560    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 21:33:02.909727 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.909690    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kfp2c\""
Sep 09 21:33:02.910559 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.910535    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:02.912850 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.912827    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 21:33:02.916141 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.916111    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 21:33:02.917029 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.916431    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6cn4p\""
Sep 09 21:33:02.917029 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.916640    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 21:33:02.917029 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.916753    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 21:33:02.917242 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.917059    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kv8vh\""
Sep 09 21:33:02.917242 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.917117    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 21:33:02.931552 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.931525    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fg8sp"
Sep 09 21:33:02.939574 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.939550    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fg8sp"
Sep 09 21:33:02.996783 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:02.996757    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 21:33:03.001950 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.001863    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal" event={"ID":"f62cbe264ea5b90cb1c40b6d5e6165e6","Type":"ContainerStarted","Data":"1be716b00c5256a9283b493ac1a9b0bfddf07d252af6fe2a9d5620720314daad"}
Sep 09 21:33:03.003524 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.003500    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-157.ec2.internal" event={"ID":"58cbb1667332061624da09e390b4d934","Type":"ContainerStarted","Data":"ce1b32d305a7f02de8f74e237e6978c24f7a2de9cf4eab2790bc01ac17b18cdb"}
Sep 09 21:33:03.007820 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.007797    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-etc-openvswitch\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.007938 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.007838    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-run-netns\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.007938 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.007864    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-run-systemd\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.007938 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.007891    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-run-ovn-kubernetes\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.007938 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.007919    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/414900e1-c678-4e61-9e76-0e2e29ac783c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:03.008192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.007950    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c297d41d-7ddc-4eaa-9982-77b1b8dbb645-agent-certs\") pod \"konnectivity-agent-m7d8c\" (UID: \"c297d41d-7ddc-4eaa-9982-77b1b8dbb645\") " pod="kube-system/konnectivity-agent-m7d8c"
Sep 09 21:33:03.008192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.007973    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-run\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.008192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-multus-socket-dir-parent\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.008192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008055    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-var-lib-cni-multus\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.008192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008086    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-var-lib-kubelet\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.008192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008116    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/37bf59e9-12ea-47c5-a476-c86019ddd720-iptables-alerter-script\") pod \"iptables-alerter-vdc7g\" (UID: \"37bf59e9-12ea-47c5-a476-c86019ddd720\") " pod="openshift-network-operator/iptables-alerter-vdc7g"
Sep 09 21:33:03.008192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-tmp\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.008192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008164    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-ovnkube-script-lib\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.008192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008191    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2v5q\" (UniqueName: \"kubernetes.io/projected/cc8427f7-1d50-457a-879b-001799103caa-kube-api-access-g2v5q\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008218    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r29c9\" (UniqueName: \"kubernetes.io/projected/a97e012d-3878-4dc9-a0c4-9f0dae4cd093-kube-api-access-r29c9\") pod \"node-ca-wcng5\" (UID: \"a97e012d-3878-4dc9-a0c4-9f0dae4cd093\") " pod="openshift-image-registry/node-ca-wcng5"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-kubernetes\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008275    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-sys\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008329    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-slash\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008354    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-cni-bin\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008371    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-run-k8s-cni-cncf-io\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008388    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-etc-selinux\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008404    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wtdl\" (UniqueName: \"kubernetes.io/projected/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-kube-api-access-6wtdl\") pod \"network-metrics-daemon-sgvmj\" (UID: \"7c340b83-fda4-4fb8-afdd-9ec6d597ec62\") " pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008421    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-var-lib-kubelet\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008447    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-system-cni-dir\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008465    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/414900e1-c678-4e61-9e76-0e2e29ac783c-system-cni-dir\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008480    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c297d41d-7ddc-4eaa-9982-77b1b8dbb645-konnectivity-ca\") pod \"konnectivity-agent-m7d8c\" (UID: \"c297d41d-7ddc-4eaa-9982-77b1b8dbb645\") " pod="kube-system/konnectivity-agent-m7d8c"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008504    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008527    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-socket-dir\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008569    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-registration-dir\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.008719 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008595    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9h76\" (UniqueName: \"kubernetes.io/projected/37bf59e9-12ea-47c5-a476-c86019ddd720-kube-api-access-d9h76\") pod \"iptables-alerter-vdc7g\" (UID: \"37bf59e9-12ea-47c5-a476-c86019ddd720\") " pod="openshift-network-operator/iptables-alerter-vdc7g"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008633    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nt62\" (UniqueName: \"kubernetes.io/projected/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-kube-api-access-4nt62\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008656    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1a022781-ea0f-4915-96be-242aac7202e6-hosts-file\") pod \"node-resolver-jrklh\" (UID: \"1a022781-ea0f-4915-96be-242aac7202e6\") " pod="openshift-dns/node-resolver-jrklh"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008687    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-systemd-units\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-env-overrides\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008761    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzf25\" (UniqueName: \"kubernetes.io/projected/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-kube-api-access-qzf25\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/37bf59e9-12ea-47c5-a476-c86019ddd720-host-slash\") pod \"iptables-alerter-vdc7g\" (UID: \"37bf59e9-12ea-47c5-a476-c86019ddd720\") " pod="openshift-network-operator/iptables-alerter-vdc7g"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008815    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnhjw\" (UniqueName: \"kubernetes.io/projected/1a022781-ea0f-4915-96be-242aac7202e6-kube-api-access-mnhjw\") pod \"node-resolver-jrklh\" (UID: \"1a022781-ea0f-4915-96be-242aac7202e6\") " pod="openshift-dns/node-resolver-jrklh"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008838    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/414900e1-c678-4e61-9e76-0e2e29ac783c-cni-binary-copy\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008854    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-modprobe-d\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008870    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-kubelet\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008893    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-var-lib-openvswitch\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008919    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-multus-cni-dir\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.008943    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cc8427f7-1d50-457a-879b-001799103caa-multus-daemon-config\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009025    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-var-lib-kubelet\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009047    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-sysctl-d\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009049    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-systemd-units\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.009535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009160    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-cni-bin\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009159    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-var-lib-cni-multus\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009199    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-lib-modules\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009229    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-host\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009252    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a97e012d-3878-4dc9-a0c4-9f0dae4cd093-host\") pod \"node-ca-wcng5\" (UID: \"a97e012d-3878-4dc9-a0c4-9f0dae4cd093\") " pod="openshift-image-registry/node-ca-wcng5"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009278    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-sys-fs\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009303    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs\") pod \"network-metrics-daemon-sgvmj\" (UID: \"7c340b83-fda4-4fb8-afdd-9ec6d597ec62\") " pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009323    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-multus-socket-dir-parent\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009349    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc8427f7-1d50-457a-879b-001799103caa-cni-binary-copy\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009383    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxtfq\" (UniqueName: \"kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq\") pod \"network-check-target-bdlgh\" (UID: \"885c3bcb-44b1-46c4-ab1c-62269c8d5474\") " pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009410    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-log-socket\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009443    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-hostroot\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009471    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-tuned\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009499    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009531    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-ovnkube-config\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009558    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nfwd\" (UniqueName: \"kubernetes.io/projected/ccfc99d1-61d2-4f0e-a156-f33c3516a479-kube-api-access-7nfwd\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009586    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-sysconfig\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.010429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009614    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-node-log\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-cni-netd\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009666    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-ovn-node-metrics-cert\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009692    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-multus-conf-dir\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009716    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-run-multus-certs\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009746    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-etc-kubernetes\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009772    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a97e012d-3878-4dc9-a0c4-9f0dae4cd093-serviceca\") pod \"node-ca-wcng5\" (UID: \"a97e012d-3878-4dc9-a0c4-9f0dae4cd093\") " pod="openshift-image-registry/node-ca-wcng5"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009798    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-systemd\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009830    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlsdv\" (UniqueName: \"kubernetes.io/projected/414900e1-c678-4e61-9e76-0e2e29ac783c-kube-api-access-nlsdv\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009860    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1995b548-a1df-4e1f-81ea-0f66bc72bf58-kubelet-config\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.009905    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/414900e1-c678-4e61-9e76-0e2e29ac783c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010121    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/414900e1-c678-4e61-9e76-0e2e29ac783c-system-cni-dir\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010125    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-etc-openvswitch\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010139    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-kubelet\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010166    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cc8427f7-1d50-457a-879b-001799103caa-multus-daemon-config\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010194    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-run-systemd\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010205    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-multus-conf-dir\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.011340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010216    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-node-log\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010229    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-system-cni-dir\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010264    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-cnibin\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-run-ovn\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010320    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-var-lib-cni-bin\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010354    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/414900e1-c678-4e61-9e76-0e2e29ac783c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010401    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-sysctl-conf\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010402    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-hostroot\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010429    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a022781-ea0f-4915-96be-242aac7202e6-tmp-dir\") pod \"node-resolver-jrklh\" (UID: \"1a022781-ea0f-4915-96be-242aac7202e6\") " pod="openshift-dns/node-resolver-jrklh"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010465    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1995b548-a1df-4e1f-81ea-0f66bc72bf58-dbus\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010473    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-etc-kubernetes\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010494    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-run-openvswitch\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010516    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-cni-netd\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010530    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010564    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-run-netns\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010580    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-multus-cni-dir\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/414900e1-c678-4e61-9e76-0e2e29ac783c-cnibin\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010617    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-run-k8s-cni-cncf-io\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.012211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/414900e1-c678-4e61-9e76-0e2e29ac783c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010658    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-ovnkube-config\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010661    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/414900e1-c678-4e61-9e76-0e2e29ac783c-cni-binary-copy\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010665    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-run-netns\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010717    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-run-multus-certs\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010771    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-var-lib-openvswitch\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010791    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/414900e1-c678-4e61-9e76-0e2e29ac783c-cnibin\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010806    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-run-ovn-kubernetes\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010829    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010858    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-cnibin\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010864    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-log-socket\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010899    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-run-openvswitch\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010906    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-run-ovn\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010938    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-run-netns\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.010980    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-host-slash\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.011032    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-device-dir\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.011129    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc8427f7-1d50-457a-879b-001799103caa-host-var-lib-cni-bin\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.011287    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc8427f7-1d50-457a-879b-001799103caa-cni-binary-copy\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.013105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.011439    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 21:33:03.013988 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.011446    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/414900e1-c678-4e61-9e76-0e2e29ac783c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:03.013988 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.011679    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/414900e1-c678-4e61-9e76-0e2e29ac783c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:03.013988 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.012267    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-env-overrides\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.013988 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.012720    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-ovnkube-script-lib\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.016460 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.016424    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-ovn-node-metrics-cert\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.020593 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.020572    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 21:33:03.020690 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.020603    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 21:33:03.020690 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.020616    2578 projected.go:196] Error preparing data for projected volume kube-api-access-pxtfq for pod openshift-network-diagnostics/network-check-target-bdlgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 21:33:03.020803 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.020701    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq podName:885c3bcb-44b1-46c4-ab1c-62269c8d5474 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:03.520657772 +0000 UTC m=+2.993622676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pxtfq" (UniqueName: "kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq") pod "network-check-target-bdlgh" (UID: "885c3bcb-44b1-46c4-ab1c-62269c8d5474") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 21:33:03.025156 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.025118    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzf25\" (UniqueName: \"kubernetes.io/projected/d7a07ed1-b2fb-48ef-b337-677f0aa1a385-kube-api-access-qzf25\") pod \"ovnkube-node-rftp9\" (UID: \"d7a07ed1-b2fb-48ef-b337-677f0aa1a385\") " pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.025391 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.025364    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2v5q\" (UniqueName: \"kubernetes.io/projected/cc8427f7-1d50-457a-879b-001799103caa-kube-api-access-g2v5q\") pod \"multus-4zq5s\" (UID: \"cc8427f7-1d50-457a-879b-001799103caa\") " pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.025802 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.025741    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlsdv\" (UniqueName: \"kubernetes.io/projected/414900e1-c678-4e61-9e76-0e2e29ac783c-kube-api-access-nlsdv\") pod \"multus-additional-cni-plugins-rqzxb\" (UID: \"414900e1-c678-4e61-9e76-0e2e29ac783c\") " pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:03.112284 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/37bf59e9-12ea-47c5-a476-c86019ddd720-host-slash\") pod \"iptables-alerter-vdc7g\" (UID: \"37bf59e9-12ea-47c5-a476-c86019ddd720\") " pod="openshift-network-operator/iptables-alerter-vdc7g"
Sep 09 21:33:03.112431 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnhjw\" (UniqueName: \"kubernetes.io/projected/1a022781-ea0f-4915-96be-242aac7202e6-kube-api-access-mnhjw\") pod \"node-resolver-jrklh\" (UID: \"1a022781-ea0f-4915-96be-242aac7202e6\") " pod="openshift-dns/node-resolver-jrklh"
Sep 09 21:33:03.112431 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112311    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-modprobe-d\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.112431 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112338    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-sysctl-d\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.112431 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112362    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-lib-modules\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.112431 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112385    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-host\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.112431 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112406    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a97e012d-3878-4dc9-a0c4-9f0dae4cd093-host\") pod \"node-ca-wcng5\" (UID: \"a97e012d-3878-4dc9-a0c4-9f0dae4cd093\") " pod="openshift-image-registry/node-ca-wcng5"
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112467    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-host\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-sys-fs\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112497    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-sysctl-d\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112502    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-lib-modules\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112526    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs\") pod \"network-metrics-daemon-sgvmj\" (UID: \"7c340b83-fda4-4fb8-afdd-9ec6d597ec62\") " pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112538    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a97e012d-3878-4dc9-a0c4-9f0dae4cd093-host\") pod \"node-ca-wcng5\" (UID: \"a97e012d-3878-4dc9-a0c4-9f0dae4cd093\") " pod="openshift-image-registry/node-ca-wcng5"
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112524    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/37bf59e9-12ea-47c5-a476-c86019ddd720-host-slash\") pod \"iptables-alerter-vdc7g\" (UID: \"37bf59e9-12ea-47c5-a476-c86019ddd720\") " pod="openshift-network-operator/iptables-alerter-vdc7g"
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112591    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-tuned\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112596    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-sys-fs\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-modprobe-d\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.112643    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112662    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nfwd\" (UniqueName: \"kubernetes.io/projected/ccfc99d1-61d2-4f0e-a156-f33c3516a479-kube-api-access-7nfwd\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112691    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-sysconfig\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.112707    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs podName:7c340b83-fda4-4fb8-afdd-9ec6d597ec62 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:03.612687136 +0000 UTC m=+3.085652026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs") pod "network-metrics-daemon-sgvmj" (UID: "7c340b83-fda4-4fb8-afdd-9ec6d597ec62") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 21:33:03.112746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112744    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a97e012d-3878-4dc9-a0c4-9f0dae4cd093-serviceca\") pod \"node-ca-wcng5\" (UID: \"a97e012d-3878-4dc9-a0c4-9f0dae4cd093\") " pod="openshift-image-registry/node-ca-wcng5"
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112748    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-sysconfig\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.112706    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112773    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-systemd\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112806    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1995b548-a1df-4e1f-81ea-0f66bc72bf58-kubelet-config\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112841    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-sysctl-conf\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112869    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a022781-ea0f-4915-96be-242aac7202e6-tmp-dir\") pod \"node-resolver-jrklh\" (UID: \"1a022781-ea0f-4915-96be-242aac7202e6\") " pod="openshift-dns/node-resolver-jrklh"
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112896    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1995b548-a1df-4e1f-81ea-0f66bc72bf58-dbus\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112899    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-systemd\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112914    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1995b548-a1df-4e1f-81ea-0f66bc72bf58-kubelet-config\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.112921    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret podName:1995b548-a1df-4e1f-81ea-0f66bc72bf58 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:03.612906149 +0000 UTC m=+3.085871054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret") pod "global-pull-secret-syncer-69ls5" (UID: "1995b548-a1df-4e1f-81ea-0f66bc72bf58") : object "kube-system"/"original-pull-secret" not registered
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.112972    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-device-dir\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113021    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c297d41d-7ddc-4eaa-9982-77b1b8dbb645-agent-certs\") pod \"konnectivity-agent-m7d8c\" (UID: \"c297d41d-7ddc-4eaa-9982-77b1b8dbb645\") " pod="kube-system/konnectivity-agent-m7d8c"
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113043    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-sysctl-conf\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113051    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1995b548-a1df-4e1f-81ea-0f66bc72bf58-dbus\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113048    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-device-dir\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113046    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-run\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.113544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113087    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-run\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113099    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/37bf59e9-12ea-47c5-a476-c86019ddd720-iptables-alerter-script\") pod \"iptables-alerter-vdc7g\" (UID: \"37bf59e9-12ea-47c5-a476-c86019ddd720\") " pod="openshift-network-operator/iptables-alerter-vdc7g"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113123    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-tmp\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r29c9\" (UniqueName: \"kubernetes.io/projected/a97e012d-3878-4dc9-a0c4-9f0dae4cd093-kube-api-access-r29c9\") pod \"node-ca-wcng5\" (UID: \"a97e012d-3878-4dc9-a0c4-9f0dae4cd093\") " pod="openshift-image-registry/node-ca-wcng5"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113177    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-kubernetes\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-sys\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113226    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-etc-selinux\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113232    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a022781-ea0f-4915-96be-242aac7202e6-tmp-dir\") pod \"node-resolver-jrklh\" (UID: \"1a022781-ea0f-4915-96be-242aac7202e6\") " pod="openshift-dns/node-resolver-jrklh"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113254    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wtdl\" (UniqueName: \"kubernetes.io/projected/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-kube-api-access-6wtdl\") pod \"network-metrics-daemon-sgvmj\" (UID: \"7c340b83-fda4-4fb8-afdd-9ec6d597ec62\") " pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113304    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-var-lib-kubelet\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113323    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-kubernetes\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113332    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c297d41d-7ddc-4eaa-9982-77b1b8dbb645-konnectivity-ca\") pod \"konnectivity-agent-m7d8c\" (UID: \"c297d41d-7ddc-4eaa-9982-77b1b8dbb645\") " pod="kube-system/konnectivity-agent-m7d8c"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113357    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113383    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-socket-dir\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113407    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-registration-dir\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113433    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9h76\" (UniqueName: \"kubernetes.io/projected/37bf59e9-12ea-47c5-a476-c86019ddd720-kube-api-access-d9h76\") pod \"iptables-alerter-vdc7g\" (UID: \"37bf59e9-12ea-47c5-a476-c86019ddd720\") " pod="openshift-network-operator/iptables-alerter-vdc7g"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113475    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nt62\" (UniqueName: \"kubernetes.io/projected/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-kube-api-access-4nt62\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.114489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113500    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1a022781-ea0f-4915-96be-242aac7202e6-hosts-file\") pod \"node-resolver-jrklh\" (UID: \"1a022781-ea0f-4915-96be-242aac7202e6\") " pod="openshift-dns/node-resolver-jrklh"
Sep 09 21:33:03.115367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113538    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a97e012d-3878-4dc9-a0c4-9f0dae4cd093-serviceca\") pod \"node-ca-wcng5\" (UID: \"a97e012d-3878-4dc9-a0c4-9f0dae4cd093\") " pod="openshift-image-registry/node-ca-wcng5"
Sep 09 21:33:03.115367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113582    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1a022781-ea0f-4915-96be-242aac7202e6-hosts-file\") pod \"node-resolver-jrklh\" (UID: \"1a022781-ea0f-4915-96be-242aac7202e6\") " pod="openshift-dns/node-resolver-jrklh"
Sep 09 21:33:03.115367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113636    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-registration-dir\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.115367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113716    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.115367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113755    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/37bf59e9-12ea-47c5-a476-c86019ddd720-iptables-alerter-script\") pod \"iptables-alerter-vdc7g\" (UID: \"37bf59e9-12ea-47c5-a476-c86019ddd720\") " pod="openshift-network-operator/iptables-alerter-vdc7g"
Sep 09 21:33:03.115367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113824    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-socket-dir\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.115367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113845    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-sys\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.115367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113920    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-var-lib-kubelet\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.115367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.113950    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c297d41d-7ddc-4eaa-9982-77b1b8dbb645-konnectivity-ca\") pod \"konnectivity-agent-m7d8c\" (UID: \"c297d41d-7ddc-4eaa-9982-77b1b8dbb645\") " pod="kube-system/konnectivity-agent-m7d8c"
Sep 09 21:33:03.115367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.114009    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ccfc99d1-61d2-4f0e-a156-f33c3516a479-etc-selinux\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.116447 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.116332    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c297d41d-7ddc-4eaa-9982-77b1b8dbb645-agent-certs\") pod \"konnectivity-agent-m7d8c\" (UID: \"c297d41d-7ddc-4eaa-9982-77b1b8dbb645\") " pod="kube-system/konnectivity-agent-m7d8c"
Sep 09 21:33:03.116447 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.116402    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-etc-tuned\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.118142 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.118123    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-tmp\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.125109 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.124856    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnhjw\" (UniqueName: \"kubernetes.io/projected/1a022781-ea0f-4915-96be-242aac7202e6-kube-api-access-mnhjw\") pod \"node-resolver-jrklh\" (UID: \"1a022781-ea0f-4915-96be-242aac7202e6\") " pod="openshift-dns/node-resolver-jrklh"
Sep 09 21:33:03.131234 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.131171    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wtdl\" (UniqueName: \"kubernetes.io/projected/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-kube-api-access-6wtdl\") pod \"network-metrics-daemon-sgvmj\" (UID: \"7c340b83-fda4-4fb8-afdd-9ec6d597ec62\") " pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:03.131234 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.131196    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nfwd\" (UniqueName: \"kubernetes.io/projected/ccfc99d1-61d2-4f0e-a156-f33c3516a479-kube-api-access-7nfwd\") pod \"aws-ebs-csi-driver-node-zrhk9\" (UID: \"ccfc99d1-61d2-4f0e-a156-f33c3516a479\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.131390 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.131226    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r29c9\" (UniqueName: \"kubernetes.io/projected/a97e012d-3878-4dc9-a0c4-9f0dae4cd093-kube-api-access-r29c9\") pod \"node-ca-wcng5\" (UID: \"a97e012d-3878-4dc9-a0c4-9f0dae4cd093\") " pod="openshift-image-registry/node-ca-wcng5"
Sep 09 21:33:03.131480 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.131459    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9h76\" (UniqueName: \"kubernetes.io/projected/37bf59e9-12ea-47c5-a476-c86019ddd720-kube-api-access-d9h76\") pod \"iptables-alerter-vdc7g\" (UID: \"37bf59e9-12ea-47c5-a476-c86019ddd720\") " pod="openshift-network-operator/iptables-alerter-vdc7g"
Sep 09 21:33:03.133210 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.133187    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nt62\" (UniqueName: \"kubernetes.io/projected/d5d16459-5d6c-4ab5-9ef2-728d76d5e491-kube-api-access-4nt62\") pod \"tuned-qbfd2\" (UID: \"d5d16459-5d6c-4ab5-9ef2-728d76d5e491\") " pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.207515 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.207479    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:03.220217 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.219891    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4zq5s"
Sep 09 21:33:03.222142 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:03.221980    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7a07ed1_b2fb_48ef_b337_677f0aa1a385.slice/crio-a26f0aa46b997e7c5d37b4fde1d7a8c9e395962638b48c39008f30d128301156 WatchSource:0}: Error finding container a26f0aa46b997e7c5d37b4fde1d7a8c9e395962638b48c39008f30d128301156: Status 404 returned error can't find the container with id a26f0aa46b997e7c5d37b4fde1d7a8c9e395962638b48c39008f30d128301156
Sep 09 21:33:03.229799 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.229779    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rqzxb"
Sep 09 21:33:03.230338 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:03.230312    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc8427f7_1d50_457a_879b_001799103caa.slice/crio-e0535ba32a4fb7964ef17b432945f4b8e16539b61ac81c3eb292f7dc418e078b WatchSource:0}: Error finding container e0535ba32a4fb7964ef17b432945f4b8e16539b61ac81c3eb292f7dc418e078b: Status 404 returned error can't find the container with id e0535ba32a4fb7964ef17b432945f4b8e16539b61ac81c3eb292f7dc418e078b
Sep 09 21:33:03.238301 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.238278    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m7d8c"
Sep 09 21:33:03.245902 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.245739    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jrklh"
Sep 09 21:33:03.248513 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:03.248484    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc297d41d_7ddc_4eaa_9982_77b1b8dbb645.slice/crio-663303c9b59e4ce1e23564158148123edddaedb788bce5a96f47a5db7bbafdd6 WatchSource:0}: Error finding container 663303c9b59e4ce1e23564158148123edddaedb788bce5a96f47a5db7bbafdd6: Status 404 returned error can't find the container with id 663303c9b59e4ce1e23564158148123edddaedb788bce5a96f47a5db7bbafdd6
Sep 09 21:33:03.255665 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.255643    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9"
Sep 09 21:33:03.256354 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:03.256327    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a022781_ea0f_4915_96be_242aac7202e6.slice/crio-6483498b53801d948e38e32a010763f663e86b46046bec0ab4f467502ced4ff3 WatchSource:0}: Error finding container 6483498b53801d948e38e32a010763f663e86b46046bec0ab4f467502ced4ff3: Status 404 returned error can't find the container with id 6483498b53801d948e38e32a010763f663e86b46046bec0ab4f467502ced4ff3
Sep 09 21:33:03.263519 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.262801    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wcng5"
Sep 09 21:33:03.273215 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.273183    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vdc7g"
Sep 09 21:33:03.275190 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:03.275130    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccfc99d1_61d2_4f0e_a156_f33c3516a479.slice/crio-2d237ed4634c06f4a49c2a2d93772d44b315313e02543aeeb2d7058a74ee0161 WatchSource:0}: Error finding container 2d237ed4634c06f4a49c2a2d93772d44b315313e02543aeeb2d7058a74ee0161: Status 404 returned error can't find the container with id 2d237ed4634c06f4a49c2a2d93772d44b315313e02543aeeb2d7058a74ee0161
Sep 09 21:33:03.278844 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.278811    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qbfd2"
Sep 09 21:33:03.279656 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:03.279630    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda97e012d_3878_4dc9_a0c4_9f0dae4cd093.slice/crio-68d176ad8647eed8da18415d3c010ab3ad92d7fca15df1846000705e87c8e7fe WatchSource:0}: Error finding container 68d176ad8647eed8da18415d3c010ab3ad92d7fca15df1846000705e87c8e7fe: Status 404 returned error can't find the container with id 68d176ad8647eed8da18415d3c010ab3ad92d7fca15df1846000705e87c8e7fe
Sep 09 21:33:03.289512 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:03.289485    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37bf59e9_12ea_47c5_a476_c86019ddd720.slice/crio-16ee806aa8c00e777ce55f72da4ed7ae5ffd2c98661b8f5e00bd943ed6f07d83 WatchSource:0}: Error finding container 16ee806aa8c00e777ce55f72da4ed7ae5ffd2c98661b8f5e00bd943ed6f07d83: Status 404 returned error can't find the container with id 16ee806aa8c00e777ce55f72da4ed7ae5ffd2c98661b8f5e00bd943ed6f07d83
Sep 09 21:33:03.293500 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:03.293470    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d16459_5d6c_4ab5_9ef2_728d76d5e491.slice/crio-127cf4f7e5bce8962cf555392eacdf0f73c8355235e18c56fa1d6a1f80e656bd WatchSource:0}: Error finding container 127cf4f7e5bce8962cf555392eacdf0f73c8355235e18c56fa1d6a1f80e656bd: Status 404 returned error can't find the container with id 127cf4f7e5bce8962cf555392eacdf0f73c8355235e18c56fa1d6a1f80e656bd
Sep 09 21:33:03.405797 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.405722    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 21:33:03.617306 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.617272    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs\") pod \"network-metrics-daemon-sgvmj\" (UID: \"7c340b83-fda4-4fb8-afdd-9ec6d597ec62\") " pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:03.617306 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.617311    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxtfq\" (UniqueName: \"kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq\") pod \"network-check-target-bdlgh\" (UID: \"885c3bcb-44b1-46c4-ab1c-62269c8d5474\") " pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:03.617556 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.617338    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:03.617556 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.617434    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 21:33:03.617556 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.617476    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 21:33:03.617556 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.617495    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs podName:7c340b83-fda4-4fb8-afdd-9ec6d597ec62 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:04.617480039 +0000 UTC m=+4.090444932 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs") pod "network-metrics-daemon-sgvmj" (UID: "7c340b83-fda4-4fb8-afdd-9ec6d597ec62") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 21:33:03.617556 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.617499    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 21:33:03.617556 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.617513    2578 projected.go:196] Error preparing data for projected volume kube-api-access-pxtfq for pod openshift-network-diagnostics/network-check-target-bdlgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 21:33:03.617556 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.617434    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 21:33:03.617556 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.617561    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret podName:1995b548-a1df-4e1f-81ea-0f66bc72bf58 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:04.61754755 +0000 UTC m=+4.090512454 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret") pod "global-pull-secret-syncer-69ls5" (UID: "1995b548-a1df-4e1f-81ea-0f66bc72bf58") : object "kube-system"/"original-pull-secret" not registered
Sep 09 21:33:03.617856 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.617578    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq podName:885c3bcb-44b1-46c4-ab1c-62269c8d5474 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:04.617569127 +0000 UTC m=+4.090534022 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pxtfq" (UniqueName: "kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq") pod "network-check-target-bdlgh" (UID: "885c3bcb-44b1-46c4-ab1c-62269c8d5474") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 21:33:03.944333 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.944293    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 21:28:03 +0000 UTC" deadline="2028-06-18 13:32:08.608496856 +0000 UTC"
Sep 09 21:33:03.944333 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.944325    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15543h59m4.664175358s"
Sep 09 21:33:03.996772 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:03.996741    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:03.996924 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:03.996889    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:04.006671 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:04.006641    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vdc7g" event={"ID":"37bf59e9-12ea-47c5-a476-c86019ddd720","Type":"ContainerStarted","Data":"16ee806aa8c00e777ce55f72da4ed7ae5ffd2c98661b8f5e00bd943ed6f07d83"}
Sep 09 21:33:04.008085 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:04.008048    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wcng5" event={"ID":"a97e012d-3878-4dc9-a0c4-9f0dae4cd093","Type":"ContainerStarted","Data":"68d176ad8647eed8da18415d3c010ab3ad92d7fca15df1846000705e87c8e7fe"}
Sep 09 21:33:04.009306 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:04.009281    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9" event={"ID":"ccfc99d1-61d2-4f0e-a156-f33c3516a479","Type":"ContainerStarted","Data":"2d237ed4634c06f4a49c2a2d93772d44b315313e02543aeeb2d7058a74ee0161"}
Sep 09 21:33:04.010417 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:04.010391    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m7d8c" event={"ID":"c297d41d-7ddc-4eaa-9982-77b1b8dbb645","Type":"ContainerStarted","Data":"663303c9b59e4ce1e23564158148123edddaedb788bce5a96f47a5db7bbafdd6"}
Sep 09 21:33:04.011461 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:04.011429    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqzxb" event={"ID":"414900e1-c678-4e61-9e76-0e2e29ac783c","Type":"ContainerStarted","Data":"58911e38ccf550ed86432f8152485644d8a911494cd48c5b72468d0a383fb0de"}
Sep 09 21:33:04.012647 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:04.012622    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4zq5s" event={"ID":"cc8427f7-1d50-457a-879b-001799103caa","Type":"ContainerStarted","Data":"e0535ba32a4fb7964ef17b432945f4b8e16539b61ac81c3eb292f7dc418e078b"}
Sep 09 21:33:04.013821 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:04.013791    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jrklh" event={"ID":"1a022781-ea0f-4915-96be-242aac7202e6","Type":"ContainerStarted","Data":"6483498b53801d948e38e32a010763f663e86b46046bec0ab4f467502ced4ff3"}
Sep 09 21:33:04.015029 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:04.015003    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9" event={"ID":"d7a07ed1-b2fb-48ef-b337-677f0aa1a385","Type":"ContainerStarted","Data":"a26f0aa46b997e7c5d37b4fde1d7a8c9e395962638b48c39008f30d128301156"}
Sep 09 21:33:04.016139 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:04.016109    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qbfd2" event={"ID":"d5d16459-5d6c-4ab5-9ef2-728d76d5e491","Type":"ContainerStarted","Data":"127cf4f7e5bce8962cf555392eacdf0f73c8355235e18c56fa1d6a1f80e656bd"}
Sep 09 21:33:04.625497 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:04.624326    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs\") pod \"network-metrics-daemon-sgvmj\" (UID: \"7c340b83-fda4-4fb8-afdd-9ec6d597ec62\") " pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:04.625497 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:04.624384    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxtfq\" (UniqueName: \"kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq\") pod \"network-check-target-bdlgh\" (UID: \"885c3bcb-44b1-46c4-ab1c-62269c8d5474\") " pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:04.625497 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:04.624416    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:04.625497 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:04.624573    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 21:33:04.625497 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:04.624637    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret podName:1995b548-a1df-4e1f-81ea-0f66bc72bf58 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:06.624618987 +0000 UTC m=+6.097583883 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret") pod "global-pull-secret-syncer-69ls5" (UID: "1995b548-a1df-4e1f-81ea-0f66bc72bf58") : object "kube-system"/"original-pull-secret" not registered
Sep 09 21:33:04.625497 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:04.625064    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 21:33:04.625497 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:04.625118    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs podName:7c340b83-fda4-4fb8-afdd-9ec6d597ec62 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:06.625100904 +0000 UTC m=+6.098065798 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs") pod "network-metrics-daemon-sgvmj" (UID: "7c340b83-fda4-4fb8-afdd-9ec6d597ec62") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 21:33:04.625497 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:04.625200    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 21:33:04.625497 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:04.625217    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 21:33:04.625497 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:04.625230    2578 projected.go:196] Error preparing data for projected volume kube-api-access-pxtfq for pod openshift-network-diagnostics/network-check-target-bdlgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 21:33:04.625497 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:04.625264    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq podName:885c3bcb-44b1-46c4-ab1c-62269c8d5474 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:06.625253201 +0000 UTC m=+6.098218097 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pxtfq" (UniqueName: "kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq") pod "network-check-target-bdlgh" (UID: "885c3bcb-44b1-46c4-ab1c-62269c8d5474") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 21:33:04.945448 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:04.945317    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 21:28:03 +0000 UTC" deadline="2028-06-21 00:04:43.69059506 +0000 UTC"
Sep 09 21:33:04.945448 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:04.945395    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15602h31m38.745204639s"
Sep 09 21:33:04.999965 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:04.999909    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:05.000154 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:05.000037    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:05.000259 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:05.000155    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:05.000259 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:05.000245    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:05.027723 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:05.027685    2578 generic.go:356] "Generic (PLEG): container finished" podID="f62cbe264ea5b90cb1c40b6d5e6165e6" containerID="e18dbe6f68620fbb49589942d0b9f051e9cb911182b6e55cfe1782bc36e0beee" exitCode=0
Sep 09 21:33:05.027858 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:05.027782    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal" event={"ID":"f62cbe264ea5b90cb1c40b6d5e6165e6","Type":"ContainerDied","Data":"e18dbe6f68620fbb49589942d0b9f051e9cb911182b6e55cfe1782bc36e0beee"}
Sep 09 21:33:05.034537 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:05.034509    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-157.ec2.internal" event={"ID":"58cbb1667332061624da09e390b4d934","Type":"ContainerStarted","Data":"e41573e852b62cbc623e29ce48df5ef543caa82db2d16cbdbcdedea14b4692b3"}
Sep 09 21:33:05.996937 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:05.996674    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:05.997381 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:05.997087    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:06.055986 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:06.055693    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal" event={"ID":"f62cbe264ea5b90cb1c40b6d5e6165e6","Type":"ContainerStarted","Data":"697551f3d651037b6b4e34b36855d9d5e6f606c1e1a5917f0ce7996b068517fe"}
Sep 09 21:33:06.070319 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:06.070265    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-157.ec2.internal" podStartSLOduration=3.070243207 podStartE2EDuration="3.070243207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:33:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:33:06.069557967 +0000 UTC m=+5.542522884" watchObservedRunningTime="2026-09-09 21:33:06.070243207 +0000 UTC m=+5.543208122"
Sep 09 21:33:06.070761 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:06.070722    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-157.ec2.internal" podStartSLOduration=4.070707512 podStartE2EDuration="4.070707512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:33:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:33:05.058340475 +0000 UTC m=+4.531305392" watchObservedRunningTime="2026-09-09 21:33:06.070707512 +0000 UTC m=+5.543672427"
Sep 09 21:33:06.642282 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:06.641644    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs\") pod \"network-metrics-daemon-sgvmj\" (UID: \"7c340b83-fda4-4fb8-afdd-9ec6d597ec62\") " pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:06.642282 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:06.641699    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxtfq\" (UniqueName: \"kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq\") pod \"network-check-target-bdlgh\" (UID: \"885c3bcb-44b1-46c4-ab1c-62269c8d5474\") " pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:06.642282 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:06.641730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:06.642282 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:06.641832    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 21:33:06.642282 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:06.641883    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 21:33:06.642282 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:06.641911    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs podName:7c340b83-fda4-4fb8-afdd-9ec6d597ec62 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:10.641890334 +0000 UTC m=+10.114855230 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs") pod "network-metrics-daemon-sgvmj" (UID: "7c340b83-fda4-4fb8-afdd-9ec6d597ec62") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 21:33:06.642282 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:06.641944    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret podName:1995b548-a1df-4e1f-81ea-0f66bc72bf58 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:10.641926784 +0000 UTC m=+10.114891692 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret") pod "global-pull-secret-syncer-69ls5" (UID: "1995b548-a1df-4e1f-81ea-0f66bc72bf58") : object "kube-system"/"original-pull-secret" not registered
Sep 09 21:33:06.642282 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:06.642049    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 21:33:06.642282 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:06.642067    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 21:33:06.642282 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:06.642080    2578 projected.go:196] Error preparing data for projected volume kube-api-access-pxtfq for pod openshift-network-diagnostics/network-check-target-bdlgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 21:33:06.642282 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:06.642121    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq podName:885c3bcb-44b1-46c4-ab1c-62269c8d5474 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:10.642107973 +0000 UTC m=+10.115072883 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pxtfq" (UniqueName: "kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq") pod "network-check-target-bdlgh" (UID: "885c3bcb-44b1-46c4-ab1c-62269c8d5474") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 21:33:06.997703 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:06.997093    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:06.997703 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:06.997239    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:06.998526 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:06.998353    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:06.998526 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:06.998472    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:07.996599 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:07.996555    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:07.996793 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:07.996725    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:08.997715 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:08.997033    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:08.997715 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:08.997101    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:08.997715 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:08.997197    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:08.997715 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:08.997667    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:09.996807 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:09.996726    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:09.996983 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:09.996881    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:10.676478 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:10.676441    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs\") pod \"network-metrics-daemon-sgvmj\" (UID: \"7c340b83-fda4-4fb8-afdd-9ec6d597ec62\") " pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:10.676982 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:10.676487    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxtfq\" (UniqueName: \"kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq\") pod \"network-check-target-bdlgh\" (UID: \"885c3bcb-44b1-46c4-ab1c-62269c8d5474\") " pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:10.676982 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:10.676523    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:10.676982 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:10.676597    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 21:33:10.676982 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:10.676628    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 21:33:10.676982 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:10.676640    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 21:33:10.676982 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:10.676662    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 21:33:10.676982 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:10.676677    2578 projected.go:196] Error preparing data for projected volume kube-api-access-pxtfq for pod openshift-network-diagnostics/network-check-target-bdlgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 21:33:10.676982 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:10.676677    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs podName:7c340b83-fda4-4fb8-afdd-9ec6d597ec62 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:18.676656645 +0000 UTC m=+18.149621539 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs") pod "network-metrics-daemon-sgvmj" (UID: "7c340b83-fda4-4fb8-afdd-9ec6d597ec62") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 21:33:10.676982 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:10.676724    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret podName:1995b548-a1df-4e1f-81ea-0f66bc72bf58 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:18.676712005 +0000 UTC m=+18.149676898 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret") pod "global-pull-secret-syncer-69ls5" (UID: "1995b548-a1df-4e1f-81ea-0f66bc72bf58") : object "kube-system"/"original-pull-secret" not registered
Sep 09 21:33:10.676982 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:10.676741    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq podName:885c3bcb-44b1-46c4-ab1c-62269c8d5474 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:18.676731241 +0000 UTC m=+18.149696133 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pxtfq" (UniqueName: "kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq") pod "network-check-target-bdlgh" (UID: "885c3bcb-44b1-46c4-ab1c-62269c8d5474") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 21:33:10.998535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:10.998455    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:10.998698 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:10.998582    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:10.998698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:10.998594    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:10.998821 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:10.998711    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:11.997310 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:11.997274    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:11.997752 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:11.997408    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:12.997313 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:12.997116    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:12.997313 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:12.997246    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:12.997919 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:12.997436    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:12.997919 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:12.997591    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:13.996981 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:13.996466    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:13.996981 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:13.996616    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:14.997454 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:14.997420    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:14.997874 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:14.997542    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:14.998018 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:14.997994    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:14.998117 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:14.998095    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:15.996986 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:15.996945    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:15.997145 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:15.997086    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:16.996601 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:16.996567    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:16.997065 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:16.996700    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:16.997065 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:16.996746    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:16.997065 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:16.996862    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:17.997200 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:17.997160    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:17.997668 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:17.997283    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:18.739339 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:18.739260    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs\") pod \"network-metrics-daemon-sgvmj\" (UID: \"7c340b83-fda4-4fb8-afdd-9ec6d597ec62\") " pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:18.739339 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:18.739309    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxtfq\" (UniqueName: \"kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq\") pod \"network-check-target-bdlgh\" (UID: \"885c3bcb-44b1-46c4-ab1c-62269c8d5474\") " pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:18.739339 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:18.739329    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:18.739598 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:18.739383    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 21:33:18.739598 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:18.739436    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 21:33:18.739598 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:18.739455    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 21:33:18.739598 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:18.739471    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 21:33:18.739598 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:18.739483    2578 projected.go:196] Error preparing data for projected volume kube-api-access-pxtfq for pod openshift-network-diagnostics/network-check-target-bdlgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 21:33:18.739598 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:18.739459    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs podName:7c340b83-fda4-4fb8-afdd-9ec6d597ec62 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:34.739439495 +0000 UTC m=+34.212404399 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs") pod "network-metrics-daemon-sgvmj" (UID: "7c340b83-fda4-4fb8-afdd-9ec6d597ec62") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 21:33:18.739598 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:18.739535    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret podName:1995b548-a1df-4e1f-81ea-0f66bc72bf58 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:34.739520076 +0000 UTC m=+34.212484981 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret") pod "global-pull-secret-syncer-69ls5" (UID: "1995b548-a1df-4e1f-81ea-0f66bc72bf58") : object "kube-system"/"original-pull-secret" not registered
Sep 09 21:33:18.739598 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:18.739550    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq podName:885c3bcb-44b1-46c4-ab1c-62269c8d5474 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:34.739540909 +0000 UTC m=+34.212505798 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pxtfq" (UniqueName: "kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq") pod "network-check-target-bdlgh" (UID: "885c3bcb-44b1-46c4-ab1c-62269c8d5474") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 21:33:18.997554 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:18.997476    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:18.997917 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:18.997607    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:18.997917 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:18.997664    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:18.997917 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:18.997774    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:19.997440 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:19.997404    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:19.997615 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:19.997558    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:20.997951 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:20.997921    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:20.998368 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:20.998036    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:20.998368 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:20.998121    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:20.998368 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:20.998230    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:21.996633 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:21.996597    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:21.996799 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:21.996732    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:22.996902 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:22.996597    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:22.997550 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:22.996661    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:22.997550 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:22.996995    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:22.997550 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:22.997056    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:23.097754 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.097724    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4zq5s" event={"ID":"cc8427f7-1d50-457a-879b-001799103caa","Type":"ContainerStarted","Data":"c50c8781c08b713966476889f2c2e9f2dae42efd7505c45dd8d78ffc2ad05ca3"}
Sep 09 21:33:23.101193 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.101160    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jrklh" event={"ID":"1a022781-ea0f-4915-96be-242aac7202e6","Type":"ContainerStarted","Data":"1ac2fefe48a94835e4c009c427fcf3c07566fd509578713bc509dcbc4af74624"}
Sep 09 21:33:23.104299 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.104267    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9" event={"ID":"d7a07ed1-b2fb-48ef-b337-677f0aa1a385","Type":"ContainerStarted","Data":"5132c0348cb4c683fe85785b6249aa770772d585b0e02be7cdea7b0986603ea5"}
Sep 09 21:33:23.104405 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.104333    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9" event={"ID":"d7a07ed1-b2fb-48ef-b337-677f0aa1a385","Type":"ContainerStarted","Data":"2a2c0108e596dabfcc28aeb09b87bb3aada6e2b146bf22e9ae2678535d2a0c16"}
Sep 09 21:33:23.104405 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.104371    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9" event={"ID":"d7a07ed1-b2fb-48ef-b337-677f0aa1a385","Type":"ContainerStarted","Data":"2bc24ddbf2bf3d2f33540647cbcf5cf2be210ce9ab4bdb7291fb702b8a7aa1d2"}
Sep 09 21:33:23.104486 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.104414    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9" event={"ID":"d7a07ed1-b2fb-48ef-b337-677f0aa1a385","Type":"ContainerStarted","Data":"a9fa6d9bc6f505e7a2972a71b9499fd31a6936ee64316a2bf80b9d4cf2d400bd"}
Sep 09 21:33:23.104486 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.104444    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9" event={"ID":"d7a07ed1-b2fb-48ef-b337-677f0aa1a385","Type":"ContainerStarted","Data":"447affe3f345091953c83acbdea179680132b0038735f6b6aa7cee6447de3871"}
Sep 09 21:33:23.104486 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.104481    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9" event={"ID":"d7a07ed1-b2fb-48ef-b337-677f0aa1a385","Type":"ContainerStarted","Data":"3480e34881b18d1b5ecf1c8903aee76090d1401bd4e30d9eaaeb07e4c9bb241f"}
Sep 09 21:33:23.105583 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.105563    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qbfd2" event={"ID":"d5d16459-5d6c-4ab5-9ef2-728d76d5e491","Type":"ContainerStarted","Data":"1ef82be8daf8126c404a91c47972c26f4b0149bda5bc4410e35481317fc144d5"}
Sep 09 21:33:23.106932 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.106912    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wcng5" event={"ID":"a97e012d-3878-4dc9-a0c4-9f0dae4cd093","Type":"ContainerStarted","Data":"f7c2bb11e3d7d4db0c4acb717b586ebbb3631f6f8e7ba11c6fb88e58e5034b8a"}
Sep 09 21:33:23.108207 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.108188    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9" event={"ID":"ccfc99d1-61d2-4f0e-a156-f33c3516a479","Type":"ContainerStarted","Data":"b444bf3ec1fd38401e73cc05397c40a4845700a3632199894bd4fc15423e4ff7"}
Sep 09 21:33:23.109425 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.109402    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m7d8c" event={"ID":"c297d41d-7ddc-4eaa-9982-77b1b8dbb645","Type":"ContainerStarted","Data":"c7907da0dceb2a9a971112d43716f1948d4e708a767255e3f014dc62de50c086"}
Sep 09 21:33:23.110659 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.110640    2578 generic.go:356] "Generic (PLEG): container finished" podID="414900e1-c678-4e61-9e76-0e2e29ac783c" containerID="001038a29e4bff804ded534c59ec270f668dfd6fa89921398755f34afba042b6" exitCode=0
Sep 09 21:33:23.110740 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.110668    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqzxb" event={"ID":"414900e1-c678-4e61-9e76-0e2e29ac783c","Type":"ContainerDied","Data":"001038a29e4bff804ded534c59ec270f668dfd6fa89921398755f34afba042b6"}
Sep 09 21:33:23.112804 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.112769    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4zq5s" podStartSLOduration=2.132266527 podStartE2EDuration="21.112757609s" totalImagesPullingTime="18.980491082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:33:23.112428815 +0000 UTC m=+22.585393725" watchObservedRunningTime="2026-09-09 21:33:23.112757609 +0000 UTC m=+22.585722518"
Sep 09 21:33:23.141717 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.141676    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m7d8c" podStartSLOduration=2.256349957 podStartE2EDuration="21.141663425s" totalImagesPullingTime="18.885313468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:33:23.141268864 +0000 UTC m=+22.614233774" watchObservedRunningTime="2026-09-09 21:33:23.141663425 +0000 UTC m=+22.614628334"
Sep 09 21:33:23.153699 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.153664    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jrklh" podStartSLOduration=2.27974988 podStartE2EDuration="21.153653035s" totalImagesPullingTime="18.873903155s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:33:23.153533186 +0000 UTC m=+22.626498084" watchObservedRunningTime="2026-09-09 21:33:23.153653035 +0000 UTC m=+22.626617944"
Sep 09 21:33:23.184075 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.184040    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qbfd2" podStartSLOduration=2.343464241 podStartE2EDuration="21.184029836s" totalImagesPullingTime="18.840565595s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:33:23.183766764 +0000 UTC m=+22.656731673" watchObservedRunningTime="2026-09-09 21:33:23.184029836 +0000 UTC m=+22.656994743"
Sep 09 21:33:23.184156 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.184141    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wcng5" podStartSLOduration=2.3328290369999998 podStartE2EDuration="21.184136792s" totalImagesPullingTime="18.851307755s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:33:23.169453981 +0000 UTC m=+22.642418892" watchObservedRunningTime="2026-09-09 21:33:23.184136792 +0000 UTC m=+22.657101703"
Sep 09 21:33:23.997134 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:23.997113    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:23.997447 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:23.997212    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:24.089183 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:24.089159    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 21:33:24.114898 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:24.114871    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vdc7g" event={"ID":"37bf59e9-12ea-47c5-a476-c86019ddd720","Type":"ContainerStarted","Data":"c6dec62ed3a9165623d186c7bdf2abb3df919188bb52bb352a373d05da2f3fc8"}
Sep 09 21:33:24.116979 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:24.116944    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9" event={"ID":"ccfc99d1-61d2-4f0e-a156-f33c3516a479","Type":"ContainerStarted","Data":"c4f377a27d3edb236802ccf0cac3a089b8a91cd5f3e333c02df92f330c306992"}
Sep 09 21:33:24.168019 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:24.167989    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m7d8c"
Sep 09 21:33:24.168446 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:24.168431    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m7d8c"
Sep 09 21:33:24.181814 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:24.181778    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vdc7g" podStartSLOduration=3.337946893 podStartE2EDuration="22.181764723s" totalImagesPullingTime="18.84381783s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:33:24.129040497 +0000 UTC m=+23.602005419" watchObservedRunningTime="2026-09-09 21:33:24.181764723 +0000 UTC m=+23.654730015"
Sep 09 21:33:24.298678 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:24.298656    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m7d8c"
Sep 09 21:33:24.299068 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:24.299053    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m7d8c"
Sep 09 21:33:24.971659 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:24.971558    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T21:33:24.089180072Z","UUID":"80cd21fd-0a73-4d80-9d4e-59472d144828","Handler":null,"Name":"","Endpoint":""}
Sep 09 21:33:24.973363 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:24.973330    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 21:33:24.973363 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:24.973363    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 21:33:24.999766 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:24.999710    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:25.000146 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:24.999816    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:25.000146 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:24.999854    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:25.000146 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:24.999932    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:25.123386 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:25.123351    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9" event={"ID":"d7a07ed1-b2fb-48ef-b337-677f0aa1a385","Type":"ContainerStarted","Data":"d50bfceb772a6b6924bea9fffd423b86c24159566cdfcc058927cbf9ebd42db6"}
Sep 09 21:33:25.996616 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:25.996589    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:25.996763 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:25.996683    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:26.127206 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:26.127161    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9" event={"ID":"ccfc99d1-61d2-4f0e-a156-f33c3516a479","Type":"ContainerStarted","Data":"ae09a04a0014271d36d3edc2d1b70f64c9930c0c90e66d87d415527d9bc30f35"}
Sep 09 21:33:26.997555 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:26.997328    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:26.997711 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:26.997413    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:26.997711 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:26.997666    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:26.997812 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:26.997754    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:27.134247 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:27.134168    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9" event={"ID":"d7a07ed1-b2fb-48ef-b337-677f0aa1a385","Type":"ContainerStarted","Data":"03eac1f03811b4f9dba043a967a3d171b6a6c88169392118207e1c58208667c8"}
Sep 09 21:33:27.158759 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:27.158708    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9" podStartSLOduration=6.191895308 podStartE2EDuration="25.158689601s" totalImagesPullingTime="18.966794293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:33:27.158385336 +0000 UTC m=+26.631350246" watchObservedRunningTime="2026-09-09 21:33:27.158689601 +0000 UTC m=+26.631654511"
Sep 09 21:33:27.158919 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:27.158899    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrhk9" podStartSLOduration=3.367296583 podStartE2EDuration="25.158892915s" totalImagesPullingTime="21.791596332s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:33:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:33:26.14288402 +0000 UTC m=+25.615848929" watchObservedRunningTime="2026-09-09 21:33:27.158892915 +0000 UTC m=+26.631857824"
Sep 09 21:33:27.996801 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:27.996772    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:27.996940 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:27.996893    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:28.137225 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:28.137196    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:28.137690 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:28.137309    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:28.137690 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:28.137510    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:28.158826 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:28.158582    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:28.158826 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:28.158822    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:33:28.997213 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:28.997145    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:28.997213 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:28.997188    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:28.997376 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:28.997294    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:28.997466 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:28.997438    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:29.006709 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:29.006680    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-69ls5"]
Sep 09 21:33:29.009878 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:29.009857    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bdlgh"]
Sep 09 21:33:29.010455 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:29.010430    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sgvmj"]
Sep 09 21:33:29.010573 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:29.010531    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:29.010660 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:29.010638    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:29.140532 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:29.140507    2578 generic.go:356] "Generic (PLEG): container finished" podID="414900e1-c678-4e61-9e76-0e2e29ac783c" containerID="369aa21bb301ec51846f696342d5332d728d1297bbb4a4fa9b8e3cdad92b0b03" exitCode=0
Sep 09 21:33:29.141055 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:29.140586    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqzxb" event={"ID":"414900e1-c678-4e61-9e76-0e2e29ac783c","Type":"ContainerDied","Data":"369aa21bb301ec51846f696342d5332d728d1297bbb4a4fa9b8e3cdad92b0b03"}
Sep 09 21:33:29.141055 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:29.140623    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:29.141055 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:29.140688    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:29.141055 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:29.140829    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:29.141055 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:29.140917    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:30.997854 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:30.997624    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:30.998215 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:30.997684    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:30.998215 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:30.997944    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:30.998215 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:30.997704    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:30.998215 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:30.998067    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:30.998215 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:30.998147    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:31.150146 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:31.150070    2578 generic.go:356] "Generic (PLEG): container finished" podID="414900e1-c678-4e61-9e76-0e2e29ac783c" containerID="2e51861cc24367b0c71fa6d3d0ae8ca72d12619a0d02e4f6b11d069f2b044bb0" exitCode=0
Sep 09 21:33:31.150146 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:31.150118    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqzxb" event={"ID":"414900e1-c678-4e61-9e76-0e2e29ac783c","Type":"ContainerDied","Data":"2e51861cc24367b0c71fa6d3d0ae8ca72d12619a0d02e4f6b11d069f2b044bb0"}
Sep 09 21:33:32.997244 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:32.997212    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:32.997648 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:32.997212    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:32.997648 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:32.997329    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:32.997648 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:32.997324    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-69ls5" podUID="1995b548-a1df-4e1f-81ea-0f66bc72bf58"
Sep 09 21:33:32.997648 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:32.997420    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bdlgh" podUID="885c3bcb-44b1-46c4-ab1c-62269c8d5474"
Sep 09 21:33:32.997648 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:32.997499    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sgvmj" podUID="7c340b83-fda4-4fb8-afdd-9ec6d597ec62"
Sep 09 21:33:33.156458 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:33.156434    2578 generic.go:356] "Generic (PLEG): container finished" podID="414900e1-c678-4e61-9e76-0e2e29ac783c" containerID="38df13ed36a08c63b6cfad1945ac25cc0dc713c1d5c85dfec0e2319feda3f759" exitCode=0
Sep 09 21:33:33.156578 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:33.156496    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqzxb" event={"ID":"414900e1-c678-4e61-9e76-0e2e29ac783c","Type":"ContainerDied","Data":"38df13ed36a08c63b6cfad1945ac25cc0dc713c1d5c85dfec0e2319feda3f759"}
Sep 09 21:33:34.757058 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.757027    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs\") pod \"network-metrics-daemon-sgvmj\" (UID: \"7c340b83-fda4-4fb8-afdd-9ec6d597ec62\") " pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:34.757444 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.757077    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxtfq\" (UniqueName: \"kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq\") pod \"network-check-target-bdlgh\" (UID: \"885c3bcb-44b1-46c4-ab1c-62269c8d5474\") " pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:34.757444 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.757106    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:34.757444 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:34.757183    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 21:33:34.757444 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:34.757204    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 21:33:34.757444 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:34.757224    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 21:33:34.757444 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:34.757233    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 21:33:34.757444 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:34.757258    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs podName:7c340b83-fda4-4fb8-afdd-9ec6d597ec62 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:06.757239386 +0000 UTC m=+66.230204274 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs") pod "network-metrics-daemon-sgvmj" (UID: "7c340b83-fda4-4fb8-afdd-9ec6d597ec62") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 21:33:34.757444 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:34.757285    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret podName:1995b548-a1df-4e1f-81ea-0f66bc72bf58 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:06.757268413 +0000 UTC m=+66.230233324 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret") pod "global-pull-secret-syncer-69ls5" (UID: "1995b548-a1df-4e1f-81ea-0f66bc72bf58") : object "kube-system"/"original-pull-secret" not registered
Sep 09 21:33:34.757444 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:34.757238    2578 projected.go:196] Error preparing data for projected volume kube-api-access-pxtfq for pod openshift-network-diagnostics/network-check-target-bdlgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 21:33:34.757444 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:34.757333    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq podName:885c3bcb-44b1-46c4-ab1c-62269c8d5474 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:06.757323619 +0000 UTC m=+66.230288526 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pxtfq" (UniqueName: "kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq") pod "network-check-target-bdlgh" (UID: "885c3bcb-44b1-46c4-ab1c-62269c8d5474") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 21:33:34.816722 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.816691    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-157.ec2.internal" event="NodeReady"
Sep 09 21:33:34.816842 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.816827    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 21:33:34.847887 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.847863    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7858557fbc-hnwxm"]
Sep 09 21:33:34.849896 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.849881    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:34.851926 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.851901    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 21:33:34.852048 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.851953    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 21:33:34.852048 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.851906    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 21:33:34.852289 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.852266    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ts5sm\""
Sep 09 21:33:34.859937 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.859913    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 21:33:34.860206 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.860186    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lcmzv"]
Sep 09 21:33:34.862239 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.862217    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:34.863625 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.863605    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7858557fbc-hnwxm"]
Sep 09 21:33:34.864098 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.864081    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-59n6x"]
Sep 09 21:33:34.864192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.864176    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 21:33:34.864254 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.864232    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 21:33:34.864400 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.864350    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k4v4f\""
Sep 09 21:33:34.865932 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.865913    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-59n6x"
Sep 09 21:33:34.867841 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.867823    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 21:33:34.867944 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.867875    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 21:33:34.867944 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.867890    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 21:33:34.868078 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.868062    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bhsnj\""
Sep 09 21:33:34.872259 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.872240    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lcmzv"]
Sep 09 21:33:34.875939 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.875484    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-59n6x"]
Sep 09 21:33:34.958469 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.958391    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:34.958469 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.958431    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-certificates\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:34.958469 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.958462    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-bound-sa-token\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:34.958674 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.958484    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:34.958674 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.958508    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjl4x\" (UniqueName: \"kubernetes.io/projected/849ad12b-744d-4efa-ace8-0ba4dec756f1-kube-api-access-cjl4x\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:34.958674 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.958606    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvkfm\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-kube-api-access-fvkfm\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:34.958674 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.958646    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-ca-trust-extracted\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:34.958845 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.958699    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/849ad12b-744d-4efa-ace8-0ba4dec756f1-tmp-dir\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:34.958845 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.958773    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-image-registry-private-configuration\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:34.958845 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.958806    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-trusted-ca\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:34.958845 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.958840    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-installation-pull-secrets\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:34.959068 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.958866    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn8f8\" (UniqueName: \"kubernetes.io/projected/21d909c2-ce09-479f-b353-8b23ce49954d-kube-api-access-bn8f8\") pod \"ingress-canary-59n6x\" (UID: \"21d909c2-ce09-479f-b353-8b23ce49954d\") " pod="openshift-ingress-canary/ingress-canary-59n6x"
Sep 09 21:33:34.959068 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.958912    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21d909c2-ce09-479f-b353-8b23ce49954d-cert\") pod \"ingress-canary-59n6x\" (UID: \"21d909c2-ce09-479f-b353-8b23ce49954d\") " pod="openshift-ingress-canary/ingress-canary-59n6x"
Sep 09 21:33:34.959068 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.958998    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/849ad12b-744d-4efa-ace8-0ba4dec756f1-config-volume\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:34.985712 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.985622    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m7d8c_c297d41d-7ddc-4eaa-9982-77b1b8dbb645/konnectivity-agent/0.log" limit=1024
Sep 09 21:33:34.997474 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.997429    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:33:34.997632 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.997429    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:33:34.997749 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:34.997645    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:33:35.000552 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.000373    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 21:33:35.000552 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.000374    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 21:33:35.000552 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.000421    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 21:33:35.000552 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.000429    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kms6v\""
Sep 09 21:33:35.000552 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.000440    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 21:33:35.000552 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.000380    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5t55x\""
Sep 09 21:33:35.060060 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060036    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-bound-sa-token\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.060166 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060068    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:35.060166 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060091    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjl4x\" (UniqueName: \"kubernetes.io/projected/849ad12b-744d-4efa-ace8-0ba4dec756f1-kube-api-access-cjl4x\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:35.060166 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060154    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvkfm\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-kube-api-access-fvkfm\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.060312 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060182    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-ca-trust-extracted\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.060312 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:35.060207    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 21:33:35.060312 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:35.060267    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls podName:849ad12b-744d-4efa-ace8-0ba4dec756f1 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:35.560251538 +0000 UTC m=+35.033216425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls") pod "dns-default-lcmzv" (UID: "849ad12b-744d-4efa-ace8-0ba4dec756f1") : secret "dns-default-metrics-tls" not found
Sep 09 21:33:35.060454 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060208    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/849ad12b-744d-4efa-ace8-0ba4dec756f1-tmp-dir\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:35.060454 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060402    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-image-registry-private-configuration\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.060454 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060436    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-trusted-ca\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.060572 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060473    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-installation-pull-secrets\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.060572 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060498    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn8f8\" (UniqueName: \"kubernetes.io/projected/21d909c2-ce09-479f-b353-8b23ce49954d-kube-api-access-bn8f8\") pod \"ingress-canary-59n6x\" (UID: \"21d909c2-ce09-479f-b353-8b23ce49954d\") " pod="openshift-ingress-canary/ingress-canary-59n6x"
Sep 09 21:33:35.060572 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060529    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21d909c2-ce09-479f-b353-8b23ce49954d-cert\") pod \"ingress-canary-59n6x\" (UID: \"21d909c2-ce09-479f-b353-8b23ce49954d\") " pod="openshift-ingress-canary/ingress-canary-59n6x"
Sep 09 21:33:35.060572 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060555    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/849ad12b-744d-4efa-ace8-0ba4dec756f1-config-volume\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:35.060748 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060575    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.060748 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060593    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-certificates\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.060843 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060821    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/849ad12b-744d-4efa-ace8-0ba4dec756f1-tmp-dir\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:35.061096 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.060864    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-ca-trust-extracted\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.061172 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:35.060917    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 21:33:35.061172 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:35.061137    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7858557fbc-hnwxm: secret "image-registry-tls" not found
Sep 09 21:33:35.061257 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.061166    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/849ad12b-744d-4efa-ace8-0ba4dec756f1-config-volume\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:35.061257 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.061190    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-certificates\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.061257 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:35.061204    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls podName:5d6373bb-187e-4ec9-9e29-fff489dc7fd8 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:35.561186376 +0000 UTC m=+35.034151285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls") pod "image-registry-7858557fbc-hnwxm" (UID: "5d6373bb-187e-4ec9-9e29-fff489dc7fd8") : secret "image-registry-tls" not found
Sep 09 21:33:35.061416 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.061400    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-trusted-ca\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.064369 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.064343    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-installation-pull-secrets\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.064471 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.064369    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21d909c2-ce09-479f-b353-8b23ce49954d-cert\") pod \"ingress-canary-59n6x\" (UID: \"21d909c2-ce09-479f-b353-8b23ce49954d\") " pod="openshift-ingress-canary/ingress-canary-59n6x"
Sep 09 21:33:35.064471 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.064413    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-image-registry-private-configuration\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.067935 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.067913    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjl4x\" (UniqueName: \"kubernetes.io/projected/849ad12b-744d-4efa-ace8-0ba4dec756f1-kube-api-access-cjl4x\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:35.068492 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.068462    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn8f8\" (UniqueName: \"kubernetes.io/projected/21d909c2-ce09-479f-b353-8b23ce49954d-kube-api-access-bn8f8\") pod \"ingress-canary-59n6x\" (UID: \"21d909c2-ce09-479f-b353-8b23ce49954d\") " pod="openshift-ingress-canary/ingress-canary-59n6x"
Sep 09 21:33:35.068851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.068826    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvkfm\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-kube-api-access-fvkfm\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.068948 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.068933    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-bound-sa-token\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.177879 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.177851    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-59n6x"
Sep 09 21:33:35.457169 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.457136    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-59n6x"]
Sep 09 21:33:35.460799 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:33:35.460771    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d909c2_ce09_479f_b353_8b23ce49954d.slice/crio-9ce110c1622b5f9465b76d794bc8bebd1be19b4e54532d122e0c2e973c28ea8d WatchSource:0}: Error finding container 9ce110c1622b5f9465b76d794bc8bebd1be19b4e54532d122e0c2e973c28ea8d: Status 404 returned error can't find the container with id 9ce110c1622b5f9465b76d794bc8bebd1be19b4e54532d122e0c2e973c28ea8d
Sep 09 21:33:35.563419 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.563391    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:35.563521 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:35.563427    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:35.563565 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:35.563533    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 21:33:35.563565 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:35.563551    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7858557fbc-hnwxm: secret "image-registry-tls" not found
Sep 09 21:33:35.563565 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:35.563551    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 21:33:35.563649 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:35.563602    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls podName:5d6373bb-187e-4ec9-9e29-fff489dc7fd8 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:36.563587168 +0000 UTC m=+36.036552085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls") pod "image-registry-7858557fbc-hnwxm" (UID: "5d6373bb-187e-4ec9-9e29-fff489dc7fd8") : secret "image-registry-tls" not found
Sep 09 21:33:35.563649 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:35.563616    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls podName:849ad12b-744d-4efa-ace8-0ba4dec756f1 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:36.563609933 +0000 UTC m=+36.036574821 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls") pod "dns-default-lcmzv" (UID: "849ad12b-744d-4efa-ace8-0ba4dec756f1") : secret "dns-default-metrics-tls" not found
Sep 09 21:33:36.164597 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:36.164555    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-59n6x" event={"ID":"21d909c2-ce09-479f-b353-8b23ce49954d","Type":"ContainerStarted","Data":"9ce110c1622b5f9465b76d794bc8bebd1be19b4e54532d122e0c2e973c28ea8d"}
Sep 09 21:33:36.572116 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:36.572077    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:36.572299 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:36.572127    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:36.572299 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:36.572231    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 21:33:36.572299 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:36.572251    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 21:33:36.572299 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:36.572275    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7858557fbc-hnwxm: secret "image-registry-tls" not found
Sep 09 21:33:36.572299 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:36.572288    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls podName:849ad12b-744d-4efa-ace8-0ba4dec756f1 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:38.572271956 +0000 UTC m=+38.045236844 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls") pod "dns-default-lcmzv" (UID: "849ad12b-744d-4efa-ace8-0ba4dec756f1") : secret "dns-default-metrics-tls" not found
Sep 09 21:33:36.572562 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:36.572331    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls podName:5d6373bb-187e-4ec9-9e29-fff489dc7fd8 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:38.572313999 +0000 UTC m=+38.045278892 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls") pod "image-registry-7858557fbc-hnwxm" (UID: "5d6373bb-187e-4ec9-9e29-fff489dc7fd8") : secret "image-registry-tls" not found
Sep 09 21:33:36.704100 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:36.704014    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m7d8c_c297d41d-7ddc-4eaa-9982-77b1b8dbb645/konnectivity-agent/0.log" limit=1024
Sep 09 21:33:37.374810 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:37.374696    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m7d8c_c297d41d-7ddc-4eaa-9982-77b1b8dbb645/konnectivity-agent/0.log" limit=1024
Sep 09 21:33:37.428192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:37.428073    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m7d8c_c297d41d-7ddc-4eaa-9982-77b1b8dbb645/konnectivity-agent/0.log" limit=1024
Sep 09 21:33:38.375865 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:38.375812    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m7d8c_c297d41d-7ddc-4eaa-9982-77b1b8dbb645/konnectivity-agent/0.log" limit=1024
Sep 09 21:33:38.430066 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:38.430031    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m7d8c_c297d41d-7ddc-4eaa-9982-77b1b8dbb645/konnectivity-agent/0.log" limit=1024
Sep 09 21:33:38.587052 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:38.587014    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:38.587052 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:38.587060    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:38.587283 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:38.587199    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 21:33:38.587283 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:38.587221    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7858557fbc-hnwxm: secret "image-registry-tls" not found
Sep 09 21:33:38.587379 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:38.587286    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 21:33:38.587379 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:38.587289    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls podName:5d6373bb-187e-4ec9-9e29-fff489dc7fd8 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:42.587270109 +0000 UTC m=+42.060235006 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls") pod "image-registry-7858557fbc-hnwxm" (UID: "5d6373bb-187e-4ec9-9e29-fff489dc7fd8") : secret "image-registry-tls" not found
Sep 09 21:33:38.587379 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:38.587353    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls podName:849ad12b-744d-4efa-ace8-0ba4dec756f1 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:42.587337013 +0000 UTC m=+42.060301907 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls") pod "dns-default-lcmzv" (UID: "849ad12b-744d-4efa-ace8-0ba4dec756f1") : secret "dns-default-metrics-tls" not found
Sep 09 21:33:39.306155 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:39.306105    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m7d8c_c297d41d-7ddc-4eaa-9982-77b1b8dbb645/konnectivity-agent/0.log" limit=1024
Sep 09 21:33:40.175633 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:40.175603    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-59n6x" event={"ID":"21d909c2-ce09-479f-b353-8b23ce49954d","Type":"ContainerStarted","Data":"8996ad15ece749e74f2817cadc77ae1bb41149fa4f6b53ffc89cd889f35d79e2"}
Sep 09 21:33:40.178489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:40.178466    2578 generic.go:356] "Generic (PLEG): container finished" podID="414900e1-c678-4e61-9e76-0e2e29ac783c" containerID="37f40711c2ede61d4bf7ecec1e0b85c810e0901f427321e70b6a667a784ced4f" exitCode=0
Sep 09 21:33:40.178595 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:40.178516    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqzxb" event={"ID":"414900e1-c678-4e61-9e76-0e2e29ac783c","Type":"ContainerDied","Data":"37f40711c2ede61d4bf7ecec1e0b85c810e0901f427321e70b6a667a784ced4f"}
Sep 09 21:33:40.191027 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:40.190987    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-59n6x" podStartSLOduration=1.7536597889999999 podStartE2EDuration="6.190958047s" totalImagesPullingTime="4.437298258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:33:40.189874709 +0000 UTC m=+39.662839618" watchObservedRunningTime="2026-09-09 21:33:40.190958047 +0000 UTC m=+39.663922960"
Sep 09 21:33:41.183986 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:41.183941    2578 generic.go:356] "Generic (PLEG): container finished" podID="414900e1-c678-4e61-9e76-0e2e29ac783c" containerID="0a70a3051ae1ac0b3c5d6185c2f514598c5763d376a55356ece597027846e299" exitCode=0
Sep 09 21:33:41.184382 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:41.184025    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqzxb" event={"ID":"414900e1-c678-4e61-9e76-0e2e29ac783c","Type":"ContainerDied","Data":"0a70a3051ae1ac0b3c5d6185c2f514598c5763d376a55356ece597027846e299"}
Sep 09 21:33:42.189894 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:42.189860    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rqzxb" event={"ID":"414900e1-c678-4e61-9e76-0e2e29ac783c","Type":"ContainerStarted","Data":"94cdc6da775c0313e13dbab65754b021bb28e383a8a353b6193f79fc483bc7b4"}
Sep 09 21:33:42.217182 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:42.217138    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rqzxb" podStartSLOduration=6.130766173 podStartE2EDuration="40.217124441s" totalImagesPullingTime="34.086358268s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:33:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:33:42.216545847 +0000 UTC m=+41.689510757" watchObservedRunningTime="2026-09-09 21:33:42.217124441 +0000 UTC m=+41.690089350"
Sep 09 21:33:42.615230 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:42.615203    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:42.615369 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:42.615236    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:42.615369 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:42.615335    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 21:33:42.615369 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:42.615339    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 21:33:42.615369 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:42.615357    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7858557fbc-hnwxm: secret "image-registry-tls" not found
Sep 09 21:33:42.615508 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:42.615379    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls podName:849ad12b-744d-4efa-ace8-0ba4dec756f1 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:50.615367691 +0000 UTC m=+50.088332583 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls") pod "dns-default-lcmzv" (UID: "849ad12b-744d-4efa-ace8-0ba4dec756f1") : secret "dns-default-metrics-tls" not found
Sep 09 21:33:42.615508 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:42.615392    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls podName:5d6373bb-187e-4ec9-9e29-fff489dc7fd8 nodeName:}" failed. No retries permitted until 2026-09-09 21:33:50.615385829 +0000 UTC m=+50.088350716 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls") pod "image-registry-7858557fbc-hnwxm" (UID: "5d6373bb-187e-4ec9-9e29-fff489dc7fd8") : secret "image-registry-tls" not found
Sep 09 21:33:50.671644 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:50.671596    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:33:50.671644 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:33:50.671649    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:33:50.672161 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:50.671757    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 21:33:50.672161 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:50.671778    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7858557fbc-hnwxm: secret "image-registry-tls" not found
Sep 09 21:33:50.672161 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:50.671783    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 21:33:50.672161 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:50.671837    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls podName:5d6373bb-187e-4ec9-9e29-fff489dc7fd8 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:06.671818512 +0000 UTC m=+66.144783403 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls") pod "image-registry-7858557fbc-hnwxm" (UID: "5d6373bb-187e-4ec9-9e29-fff489dc7fd8") : secret "image-registry-tls" not found
Sep 09 21:33:50.672161 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:33:50.671853    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls podName:849ad12b-744d-4efa-ace8-0ba4dec756f1 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:06.671847126 +0000 UTC m=+66.144812014 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls") pod "dns-default-lcmzv" (UID: "849ad12b-744d-4efa-ace8-0ba4dec756f1") : secret "dns-default-metrics-tls" not found
Sep 09 21:34:00.154733 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:00.154704    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rftp9"
Sep 09 21:34:05.612265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.612233    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-mb2cr"]
Sep 09 21:34:05.614736 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.614719    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr"
Sep 09 21:34:05.616777 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.616756    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4dgrs\""
Sep 09 21:34:05.617199 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.617179    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 09 21:34:05.617469 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.617455    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 09 21:34:05.629119 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.629096    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-mb2cr"]
Sep 09 21:34:05.671637 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.671612    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ee4873ec-62e9-4389-bc83-7b2736c20d08-nginx-conf\") pod \"networking-console-plugin-79977f9844-mb2cr\" (UID: \"ee4873ec-62e9-4389-bc83-7b2736c20d08\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr"
Sep 09 21:34:05.671740 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.671641    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mb2cr\" (UID: \"ee4873ec-62e9-4389-bc83-7b2736c20d08\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr"
Sep 09 21:34:05.707852 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.707829    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rhkfk"]
Sep 09 21:34:05.709679 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.709665    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rhkfk"
Sep 09 21:34:05.713859 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.713832    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kjfxm\""
Sep 09 21:34:05.713998 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.713832    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 09 21:34:05.713998 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.713882    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 09 21:34:05.718346 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.718323    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rhkfk"]
Sep 09 21:34:05.772716 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.772690    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ee4873ec-62e9-4389-bc83-7b2736c20d08-nginx-conf\") pod \"networking-console-plugin-79977f9844-mb2cr\" (UID: \"ee4873ec-62e9-4389-bc83-7b2736c20d08\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr"
Sep 09 21:34:05.772808 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.772719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mb2cr\" (UID: \"ee4873ec-62e9-4389-bc83-7b2736c20d08\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr"
Sep 09 21:34:05.772808 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.772749    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck7gd\" (UniqueName: \"kubernetes.io/projected/2f5dc97e-22f3-457d-bc7e-ad1e262691ba-kube-api-access-ck7gd\") pod \"volume-data-source-validator-695c895bd6-rhkfk\" (UID: \"2f5dc97e-22f3-457d-bc7e-ad1e262691ba\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rhkfk"
Sep 09 21:34:05.772877 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:05.772852    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 21:34:05.772943 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:05.772931    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert podName:ee4873ec-62e9-4389-bc83-7b2736c20d08 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:06.272911062 +0000 UTC m=+65.745875963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mb2cr" (UID: "ee4873ec-62e9-4389-bc83-7b2736c20d08") : secret "networking-console-plugin-cert" not found
Sep 09 21:34:05.773360 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.773344    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ee4873ec-62e9-4389-bc83-7b2736c20d08-nginx-conf\") pod \"networking-console-plugin-79977f9844-mb2cr\" (UID: \"ee4873ec-62e9-4389-bc83-7b2736c20d08\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr"
Sep 09 21:34:05.808585 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.808562    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn"]
Sep 09 21:34:05.810681 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.810666    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn"
Sep 09 21:34:05.812847 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.812819    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 09 21:34:05.813010 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.812891    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-nqzzc\""
Sep 09 21:34:05.813010 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.812915    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 09 21:34:05.813010 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.812922    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 09 21:34:05.813220 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.813204    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 09 21:34:05.819168 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.819150    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-mksnd"]
Sep 09 21:34:05.821073 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.821055    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7685746588-wcx7f"]
Sep 09 21:34:05.821180 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.821165    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:05.822686 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.822670    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn"]
Sep 09 21:34:05.822774 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.822734    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:05.823294 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.823272    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 09 21:34:05.823596 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.823436    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 09 21:34:05.823596 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.823487    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-tmczt\""
Sep 09 21:34:05.823596 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.823488    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 09 21:34:05.823596 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.823534    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 09 21:34:05.824819 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.824801    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 09 21:34:05.824999 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.824980    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 09 21:34:05.825590 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.825567    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 09 21:34:05.825680 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.825603    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 09 21:34:05.825739 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.825721    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 09 21:34:05.825834 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.825817    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 09 21:34:05.825984 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.825955    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zbt69\""
Sep 09 21:34:05.829814 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.829790    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-mksnd"]
Sep 09 21:34:05.830243 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.830225    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 09 21:34:05.831597 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.831578    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7685746588-wcx7f"]
Sep 09 21:34:05.873057 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.873003    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/226b4308-cb75-45b3-aaa6-467856e4d5eb-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-6nvrn\" (UID: \"226b4308-cb75-45b3-aaa6-467856e4d5eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn"
Sep 09 21:34:05.873057 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.873032    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-default-certificate\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:05.873057 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.873052    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:05.873209 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.873077    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-stats-auth\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:05.873209 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.873102    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:05.873209 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.873118    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvcfz\" (UniqueName: \"kubernetes.io/projected/ab94f25b-423a-42a4-8784-2004122e692d-kube-api-access-hvcfz\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:05.873209 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.873165    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twp6s\" (UniqueName: \"kubernetes.io/projected/14f44884-6479-48c1-9961-717810d01fc8-kube-api-access-twp6s\") pod \"console-operator-85b454874b-mksnd\" (UID: \"14f44884-6479-48c1-9961-717810d01fc8\") " pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:05.873332 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.873210    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/14f44884-6479-48c1-9961-717810d01fc8-trusted-ca\") pod \"console-operator-85b454874b-mksnd\" (UID: \"14f44884-6479-48c1-9961-717810d01fc8\") " pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:05.873332 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.873255    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/226b4308-cb75-45b3-aaa6-467856e4d5eb-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-6nvrn\" (UID: \"226b4308-cb75-45b3-aaa6-467856e4d5eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn"
Sep 09 21:34:05.873332 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.873281    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck7gd\" (UniqueName: \"kubernetes.io/projected/2f5dc97e-22f3-457d-bc7e-ad1e262691ba-kube-api-access-ck7gd\") pod \"volume-data-source-validator-695c895bd6-rhkfk\" (UID: \"2f5dc97e-22f3-457d-bc7e-ad1e262691ba\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rhkfk"
Sep 09 21:34:05.873332 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.873296    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/14f44884-6479-48c1-9961-717810d01fc8-config\") pod \"console-operator-85b454874b-mksnd\" (UID: \"14f44884-6479-48c1-9961-717810d01fc8\") " pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:05.873332 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.873313    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/14f44884-6479-48c1-9961-717810d01fc8-serving-cert\") pod \"console-operator-85b454874b-mksnd\" (UID: \"14f44884-6479-48c1-9961-717810d01fc8\") " pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:05.873611 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.873337    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccvj7\" (UniqueName: \"kubernetes.io/projected/226b4308-cb75-45b3-aaa6-467856e4d5eb-kube-api-access-ccvj7\") pod \"kube-storage-version-migrator-operator-8d7dd4675-6nvrn\" (UID: \"226b4308-cb75-45b3-aaa6-467856e4d5eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn"
Sep 09 21:34:05.892986 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.890495    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck7gd\" (UniqueName: \"kubernetes.io/projected/2f5dc97e-22f3-457d-bc7e-ad1e262691ba-kube-api-access-ck7gd\") pod \"volume-data-source-validator-695c895bd6-rhkfk\" (UID: \"2f5dc97e-22f3-457d-bc7e-ad1e262691ba\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rhkfk"
Sep 09 21:34:05.974188 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.974158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/226b4308-cb75-45b3-aaa6-467856e4d5eb-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-6nvrn\" (UID: \"226b4308-cb75-45b3-aaa6-467856e4d5eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn"
Sep 09 21:34:05.974300 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.974201    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/14f44884-6479-48c1-9961-717810d01fc8-config\") pod \"console-operator-85b454874b-mksnd\" (UID: \"14f44884-6479-48c1-9961-717810d01fc8\") " pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:05.974300 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.974218    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/14f44884-6479-48c1-9961-717810d01fc8-serving-cert\") pod \"console-operator-85b454874b-mksnd\" (UID: \"14f44884-6479-48c1-9961-717810d01fc8\") " pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:05.974300 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.974239    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccvj7\" (UniqueName: \"kubernetes.io/projected/226b4308-cb75-45b3-aaa6-467856e4d5eb-kube-api-access-ccvj7\") pod \"kube-storage-version-migrator-operator-8d7dd4675-6nvrn\" (UID: \"226b4308-cb75-45b3-aaa6-467856e4d5eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn"
Sep 09 21:34:05.974300 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.974292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/226b4308-cb75-45b3-aaa6-467856e4d5eb-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-6nvrn\" (UID: \"226b4308-cb75-45b3-aaa6-467856e4d5eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn"
Sep 09 21:34:05.974499 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.974317    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-default-certificate\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:05.974499 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.974343    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:05.974499 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.974368    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-stats-auth\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:05.974499 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.974393    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:05.974499 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:05.974470    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 21:34:05.974736 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:05.974526    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs podName:ab94f25b-423a-42a4-8784-2004122e692d nodeName:}" failed. No retries permitted until 2026-09-09 21:34:06.474506668 +0000 UTC m=+65.947471559 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs") pod "router-default-7685746588-wcx7f" (UID: "ab94f25b-423a-42a4-8784-2004122e692d") : secret "router-metrics-certs-default" not found
Sep 09 21:34:05.974736 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:05.974704    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle podName:ab94f25b-423a-42a4-8784-2004122e692d nodeName:}" failed. No retries permitted until 2026-09-09 21:34:06.474678954 +0000 UTC m=+65.947643873 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle") pod "router-default-7685746588-wcx7f" (UID: "ab94f25b-423a-42a4-8784-2004122e692d") : configmap references non-existent config key: service-ca.crt
Sep 09 21:34:05.974863 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.974739    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvcfz\" (UniqueName: \"kubernetes.io/projected/ab94f25b-423a-42a4-8784-2004122e692d-kube-api-access-hvcfz\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:05.974863 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.974813    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twp6s\" (UniqueName: \"kubernetes.io/projected/14f44884-6479-48c1-9961-717810d01fc8-kube-api-access-twp6s\") pod \"console-operator-85b454874b-mksnd\" (UID: \"14f44884-6479-48c1-9961-717810d01fc8\") " pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:05.974863 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.974844    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/14f44884-6479-48c1-9961-717810d01fc8-trusted-ca\") pod \"console-operator-85b454874b-mksnd\" (UID: \"14f44884-6479-48c1-9961-717810d01fc8\") " pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:05.975039 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.974986    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/14f44884-6479-48c1-9961-717810d01fc8-config\") pod \"console-operator-85b454874b-mksnd\" (UID: \"14f44884-6479-48c1-9961-717810d01fc8\") " pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:05.975039 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.975029    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/226b4308-cb75-45b3-aaa6-467856e4d5eb-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-6nvrn\" (UID: \"226b4308-cb75-45b3-aaa6-467856e4d5eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn"
Sep 09 21:34:05.975788 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.975760    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/14f44884-6479-48c1-9961-717810d01fc8-trusted-ca\") pod \"console-operator-85b454874b-mksnd\" (UID: \"14f44884-6479-48c1-9961-717810d01fc8\") " pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:05.976641 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.976623    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/226b4308-cb75-45b3-aaa6-467856e4d5eb-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-6nvrn\" (UID: \"226b4308-cb75-45b3-aaa6-467856e4d5eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn"
Sep 09 21:34:05.976947 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.976928    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-stats-auth\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:05.977021 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.977005    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-default-certificate\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:05.977345 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.977329    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/14f44884-6479-48c1-9961-717810d01fc8-serving-cert\") pod \"console-operator-85b454874b-mksnd\" (UID: \"14f44884-6479-48c1-9961-717810d01fc8\") " pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:05.982039 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.982013    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccvj7\" (UniqueName: \"kubernetes.io/projected/226b4308-cb75-45b3-aaa6-467856e4d5eb-kube-api-access-ccvj7\") pod \"kube-storage-version-migrator-operator-8d7dd4675-6nvrn\" (UID: \"226b4308-cb75-45b3-aaa6-467856e4d5eb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn"
Sep 09 21:34:05.983701 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.983674    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twp6s\" (UniqueName: \"kubernetes.io/projected/14f44884-6479-48c1-9961-717810d01fc8-kube-api-access-twp6s\") pod \"console-operator-85b454874b-mksnd\" (UID: \"14f44884-6479-48c1-9961-717810d01fc8\") " pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:05.983898 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:05.983880    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvcfz\" (UniqueName: \"kubernetes.io/projected/ab94f25b-423a-42a4-8784-2004122e692d-kube-api-access-hvcfz\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:06.022873 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.022843    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rhkfk"
Sep 09 21:34:06.119140 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.119108    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn"
Sep 09 21:34:06.132295 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.132268    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:06.277181 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.277151    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mb2cr\" (UID: \"ee4873ec-62e9-4389-bc83-7b2736c20d08\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr"
Sep 09 21:34:06.277307 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:06.277288    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 21:34:06.277360 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:06.277351    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert podName:ee4873ec-62e9-4389-bc83-7b2736c20d08 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:07.277335456 +0000 UTC m=+66.750300343 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mb2cr" (UID: "ee4873ec-62e9-4389-bc83-7b2736c20d08") : secret "networking-console-plugin-cert" not found
Sep 09 21:34:06.290437 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.290412    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rhkfk"]
Sep 09 21:34:06.295022 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:06.295003    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f5dc97e_22f3_457d_bc7e_ad1e262691ba.slice/crio-dfb2d5127afbaf53de36dad5022361df46eedbbe44432de1a93363b0fdb02a5b WatchSource:0}: Error finding container dfb2d5127afbaf53de36dad5022361df46eedbbe44432de1a93363b0fdb02a5b: Status 404 returned error can't find the container with id dfb2d5127afbaf53de36dad5022361df46eedbbe44432de1a93363b0fdb02a5b
Sep 09 21:34:06.383765 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.383696    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn"]
Sep 09 21:34:06.386829 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:06.386808    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod226b4308_cb75_45b3_aaa6_467856e4d5eb.slice/crio-db7e6fd848702f4f7e1f2e81b55f72511e169492061008996227fff85916f80e WatchSource:0}: Error finding container db7e6fd848702f4f7e1f2e81b55f72511e169492061008996227fff85916f80e: Status 404 returned error can't find the container with id db7e6fd848702f4f7e1f2e81b55f72511e169492061008996227fff85916f80e
Sep 09 21:34:06.404291 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.404270    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-mksnd"]
Sep 09 21:34:06.406870 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:06.406849    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14f44884_6479_48c1_9961_717810d01fc8.slice/crio-7f18403b198baabd31de0b9c19a1823dfdd4d6f82a6a90ffb46b739dd895704d WatchSource:0}: Error finding container 7f18403b198baabd31de0b9c19a1823dfdd4d6f82a6a90ffb46b739dd895704d: Status 404 returned error can't find the container with id 7f18403b198baabd31de0b9c19a1823dfdd4d6f82a6a90ffb46b739dd895704d
Sep 09 21:34:06.478665 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.478639    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:06.478750 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.478668    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:06.478791 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:06.478782    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle podName:ab94f25b-423a-42a4-8784-2004122e692d nodeName:}" failed. No retries permitted until 2026-09-09 21:34:07.47876713 +0000 UTC m=+66.951732018 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle") pod "router-default-7685746588-wcx7f" (UID: "ab94f25b-423a-42a4-8784-2004122e692d") : configmap references non-existent config key: service-ca.crt
Sep 09 21:34:06.478835 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:06.478827    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 21:34:06.478884 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:06.478874    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs podName:ab94f25b-423a-42a4-8784-2004122e692d nodeName:}" failed. No retries permitted until 2026-09-09 21:34:07.478862692 +0000 UTC m=+66.951827580 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs") pod "router-default-7685746588-wcx7f" (UID: "ab94f25b-423a-42a4-8784-2004122e692d") : secret "router-metrics-certs-default" not found
Sep 09 21:34:06.680932 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.680855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:34:06.680932 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.680903    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:34:06.681305 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:06.681025    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 21:34:06.681305 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:06.681042    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7858557fbc-hnwxm: secret "image-registry-tls" not found
Sep 09 21:34:06.681305 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:06.681072    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 21:34:06.681305 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:06.681103    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls podName:5d6373bb-187e-4ec9-9e29-fff489dc7fd8 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:38.681088099 +0000 UTC m=+98.154052987 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls") pod "image-registry-7858557fbc-hnwxm" (UID: "5d6373bb-187e-4ec9-9e29-fff489dc7fd8") : secret "image-registry-tls" not found
Sep 09 21:34:06.681305 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:06.681127    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls podName:849ad12b-744d-4efa-ace8-0ba4dec756f1 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:38.681114857 +0000 UTC m=+98.154079769 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls") pod "dns-default-lcmzv" (UID: "849ad12b-744d-4efa-ace8-0ba4dec756f1") : secret "dns-default-metrics-tls" not found
Sep 09 21:34:06.781764 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.781736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs\") pod \"network-metrics-daemon-sgvmj\" (UID: \"7c340b83-fda4-4fb8-afdd-9ec6d597ec62\") " pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:34:06.781886 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.781791    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxtfq\" (UniqueName: \"kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq\") pod \"network-check-target-bdlgh\" (UID: \"885c3bcb-44b1-46c4-ab1c-62269c8d5474\") " pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:34:06.781886 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.781807    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:34:06.784363 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.784342    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 21:34:06.784475 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.784457    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 21:34:06.784540 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.784469    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 21:34:06.792114 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:06.792094    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 21:34:06.792209 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:06.792160    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs podName:7c340b83-fda4-4fb8-afdd-9ec6d597ec62 nodeName:}" failed. No retries permitted until 2026-09-09 21:35:10.792142393 +0000 UTC m=+130.265107287 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs") pod "network-metrics-daemon-sgvmj" (UID: "7c340b83-fda4-4fb8-afdd-9ec6d597ec62") : secret "metrics-daemon-secret" not found
Sep 09 21:34:06.793653 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.793635    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 21:34:06.794281 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.794263    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1995b548-a1df-4e1f-81ea-0f66bc72bf58-original-pull-secret\") pod \"global-pull-secret-syncer-69ls5\" (UID: \"1995b548-a1df-4e1f-81ea-0f66bc72bf58\") " pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:34:06.805029 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.805012    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxtfq\" (UniqueName: \"kubernetes.io/projected/885c3bcb-44b1-46c4-ab1c-62269c8d5474-kube-api-access-pxtfq\") pod \"network-check-target-bdlgh\" (UID: \"885c3bcb-44b1-46c4-ab1c-62269c8d5474\") " pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:34:06.808778 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.808763    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-69ls5"
Sep 09 21:34:06.818108 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.818090    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kms6v\""
Sep 09 21:34:06.826983 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:06.826951    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:34:07.077572 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:07.077497    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-69ls5"]
Sep 09 21:34:07.082023 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:07.081062    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1995b548_a1df_4e1f_81ea_0f66bc72bf58.slice/crio-631fdae5a110271d302bc6d871189aec599fd548495420156fa593011e9b3f26 WatchSource:0}: Error finding container 631fdae5a110271d302bc6d871189aec599fd548495420156fa593011e9b3f26: Status 404 returned error can't find the container with id 631fdae5a110271d302bc6d871189aec599fd548495420156fa593011e9b3f26
Sep 09 21:34:07.096253 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:07.096190    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bdlgh"]
Sep 09 21:34:07.100286 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:07.100254    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod885c3bcb_44b1_46c4_ab1c_62269c8d5474.slice/crio-5bc084c08e81bd84a657cb0ec0657171a0eb462dc97e86c11ffd796568531753 WatchSource:0}: Error finding container 5bc084c08e81bd84a657cb0ec0657171a0eb462dc97e86c11ffd796568531753: Status 404 returned error can't find the container with id 5bc084c08e81bd84a657cb0ec0657171a0eb462dc97e86c11ffd796568531753
Sep 09 21:34:07.251330 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:07.251238    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-69ls5" event={"ID":"1995b548-a1df-4e1f-81ea-0f66bc72bf58","Type":"ContainerStarted","Data":"631fdae5a110271d302bc6d871189aec599fd548495420156fa593011e9b3f26"}
Sep 09 21:34:07.252927 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:07.252877    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rhkfk" event={"ID":"2f5dc97e-22f3-457d-bc7e-ad1e262691ba","Type":"ContainerStarted","Data":"dfb2d5127afbaf53de36dad5022361df46eedbbe44432de1a93363b0fdb02a5b"}
Sep 09 21:34:07.254907 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:07.254849    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bdlgh" event={"ID":"885c3bcb-44b1-46c4-ab1c-62269c8d5474","Type":"ContainerStarted","Data":"5bc084c08e81bd84a657cb0ec0657171a0eb462dc97e86c11ffd796568531753"}
Sep 09 21:34:07.256685 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:07.256655    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-mksnd" event={"ID":"14f44884-6479-48c1-9961-717810d01fc8","Type":"ContainerStarted","Data":"7f18403b198baabd31de0b9c19a1823dfdd4d6f82a6a90ffb46b739dd895704d"}
Sep 09 21:34:07.258411 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:07.258352    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn" event={"ID":"226b4308-cb75-45b3-aaa6-467856e4d5eb","Type":"ContainerStarted","Data":"db7e6fd848702f4f7e1f2e81b55f72511e169492061008996227fff85916f80e"}
Sep 09 21:34:07.285567 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:07.285516    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mb2cr\" (UID: \"ee4873ec-62e9-4389-bc83-7b2736c20d08\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr"
Sep 09 21:34:07.285719 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:07.285698    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 21:34:07.285787 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:07.285777    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert podName:ee4873ec-62e9-4389-bc83-7b2736c20d08 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:09.285755799 +0000 UTC m=+68.758720694 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mb2cr" (UID: "ee4873ec-62e9-4389-bc83-7b2736c20d08") : secret "networking-console-plugin-cert" not found
Sep 09 21:34:07.488077 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:07.487713    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:07.488077 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:07.487770    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:07.488077 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:07.487872    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle podName:ab94f25b-423a-42a4-8784-2004122e692d nodeName:}" failed. No retries permitted until 2026-09-09 21:34:09.487849724 +0000 UTC m=+68.960814615 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle") pod "router-default-7685746588-wcx7f" (UID: "ab94f25b-423a-42a4-8784-2004122e692d") : configmap references non-existent config key: service-ca.crt
Sep 09 21:34:07.488077 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:07.487938    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 21:34:07.488077 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:07.488009    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs podName:ab94f25b-423a-42a4-8784-2004122e692d nodeName:}" failed. No retries permitted until 2026-09-09 21:34:09.487992214 +0000 UTC m=+68.960957118 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs") pod "router-default-7685746588-wcx7f" (UID: "ab94f25b-423a-42a4-8784-2004122e692d") : secret "router-metrics-certs-default" not found
Sep 09 21:34:08.264399 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:08.264364    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rhkfk" event={"ID":"2f5dc97e-22f3-457d-bc7e-ad1e262691ba","Type":"ContainerStarted","Data":"67b64683793bb10bd5db1b7d03dd46e77405226dd45708540cb31001af85ea22"}
Sep 09 21:34:08.278047 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:08.278005    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rhkfk" podStartSLOduration=1.793042914 podStartE2EDuration="3.277985911s" totalImagesPullingTime="1.484942997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:34:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:34:08.277484072 +0000 UTC m=+67.750449008" watchObservedRunningTime="2026-09-09 21:34:08.277985911 +0000 UTC m=+67.750950831"
Sep 09 21:34:09.301133 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:09.301099    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mb2cr\" (UID: \"ee4873ec-62e9-4389-bc83-7b2736c20d08\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr"
Sep 09 21:34:09.301743 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:09.301642    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 21:34:09.301743 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:09.301713    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert podName:ee4873ec-62e9-4389-bc83-7b2736c20d08 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:13.301693805 +0000 UTC m=+72.774658698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mb2cr" (UID: "ee4873ec-62e9-4389-bc83-7b2736c20d08") : secret "networking-console-plugin-cert" not found
Sep 09 21:34:09.502220 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:09.502185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:09.502370 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:09.502229    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:09.502444 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:09.502371    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle podName:ab94f25b-423a-42a4-8784-2004122e692d nodeName:}" failed. No retries permitted until 2026-09-09 21:34:13.502349377 +0000 UTC m=+72.975314280 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle") pod "router-default-7685746588-wcx7f" (UID: "ab94f25b-423a-42a4-8784-2004122e692d") : configmap references non-existent config key: service-ca.crt
Sep 09 21:34:09.502444 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:09.502382    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 21:34:09.502444 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:09.502427    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs podName:ab94f25b-423a-42a4-8784-2004122e692d nodeName:}" failed. No retries permitted until 2026-09-09 21:34:13.502416048 +0000 UTC m=+72.975380950 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs") pod "router-default-7685746588-wcx7f" (UID: "ab94f25b-423a-42a4-8784-2004122e692d") : secret "router-metrics-certs-default" not found
Sep 09 21:34:10.273760 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:10.273723    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-mksnd" event={"ID":"14f44884-6479-48c1-9961-717810d01fc8","Type":"ContainerStarted","Data":"125f7ecdc7e71ff88a7b97a0d3a7e033c7797244e484a4c37675cb53ce38a407"}
Sep 09 21:34:10.274515 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:10.274497    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:10.275994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:10.275770    2578 patch_prober.go:28] interesting pod/console-operator-85b454874b-mksnd container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused" start-of-body=
Sep 09 21:34:10.275994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:10.275824    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-85b454874b-mksnd" podUID="14f44884-6479-48c1-9961-717810d01fc8" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused"
Sep 09 21:34:10.276839 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:10.276776    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn" event={"ID":"226b4308-cb75-45b3-aaa6-467856e4d5eb","Type":"ContainerStarted","Data":"21f103be882c898305a63a52f624cb51897ed0964b346d2cbd381921501037be"}
Sep 09 21:34:10.291570 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:10.291520    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-mksnd" podStartSLOduration=2.31800939 podStartE2EDuration="5.291500085s" totalImagesPullingTime="2.973490695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:34:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:34:10.290049897 +0000 UTC m=+69.763014808" watchObservedRunningTime="2026-09-09 21:34:10.291500085 +0000 UTC m=+69.764464996"
Sep 09 21:34:10.304048 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:10.304000    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn" podStartSLOduration=2.307532306 podStartE2EDuration="5.303983068s" totalImagesPullingTime="2.996450762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:34:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:34:10.303008773 +0000 UTC m=+69.775973687" watchObservedRunningTime="2026-09-09 21:34:10.303983068 +0000 UTC m=+69.776947971"
Sep 09 21:34:11.281258 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.281229    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/0.log"
Sep 09 21:34:11.281520 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.281272    2578 generic.go:356] "Generic (PLEG): container finished" podID="14f44884-6479-48c1-9961-717810d01fc8" containerID="125f7ecdc7e71ff88a7b97a0d3a7e033c7797244e484a4c37675cb53ce38a407" exitCode=255
Sep 09 21:34:11.281520 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.281368    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-mksnd" event={"ID":"14f44884-6479-48c1-9961-717810d01fc8","Type":"ContainerDied","Data":"125f7ecdc7e71ff88a7b97a0d3a7e033c7797244e484a4c37675cb53ce38a407"}
Sep 09 21:34:11.281675 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.281661    2578 scope.go:117] "RemoveContainer" containerID="125f7ecdc7e71ff88a7b97a0d3a7e033c7797244e484a4c37675cb53ce38a407"
Sep 09 21:34:11.595999 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.595905    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mwjgp"]
Sep 09 21:34:11.598219 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.598190    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:11.600477 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.600395    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 21:34:11.600619 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.600513    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 21:34:11.600619 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.600592    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 21:34:11.600730 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.600711    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-b8s76\""
Sep 09 21:34:11.600784 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.600741    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 21:34:11.608834 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.608813    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mwjgp"]
Sep 09 21:34:11.618176 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.618155    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:11.618303 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.618183    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e282a8be-3ba4-42d0-ade2-241c1cf8a144-crio-socket\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:11.618303 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.618206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xqkq\" (UniqueName: \"kubernetes.io/projected/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-api-access-4xqkq\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:11.618419 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.618314    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e282a8be-3ba4-42d0-ade2-241c1cf8a144-data-volume\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:11.618419 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.618374    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:11.719642 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.719610    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e282a8be-3ba4-42d0-ade2-241c1cf8a144-data-volume\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:11.719827 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.719674    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:11.719827 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.719739    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:11.719939 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:11.719829    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 21:34:11.719939 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.719862    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e282a8be-3ba4-42d0-ade2-241c1cf8a144-crio-socket\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:11.719939 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.719893    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xqkq\" (UniqueName: \"kubernetes.io/projected/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-api-access-4xqkq\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:11.719939 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:11.719911    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm podName:e282a8be-3ba4-42d0-ade2-241c1cf8a144 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:12.219889229 +0000 UTC m=+71.692854156 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mwjgp" (UID: "e282a8be-3ba4-42d0-ade2-241c1cf8a144") : configmap "kube-rbac-proxy" not found
Sep 09 21:34:11.720166 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:11.719835    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 21:34:11.720166 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.720012    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e282a8be-3ba4-42d0-ade2-241c1cf8a144-data-volume\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:11.720166 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:11.720054    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls podName:e282a8be-3ba4-42d0-ade2-241c1cf8a144 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:12.220040792 +0000 UTC m=+71.693005709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mwjgp" (UID: "e282a8be-3ba4-42d0-ade2-241c1cf8a144") : secret "insights-runtime-extractor-tls" not found
Sep 09 21:34:11.720166 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.720125    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e282a8be-3ba4-42d0-ade2-241c1cf8a144-crio-socket\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:11.727875 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:11.727847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xqkq\" (UniqueName: \"kubernetes.io/projected/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-api-access-4xqkq\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:12.224518 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:12.224487    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:12.224640 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:12.224589    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:12.224692 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:12.224638    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 21:34:12.224692 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:12.224685    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 21:34:12.224765 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:12.224704    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls podName:e282a8be-3ba4-42d0-ade2-241c1cf8a144 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:13.224687272 +0000 UTC m=+72.697652177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mwjgp" (UID: "e282a8be-3ba4-42d0-ade2-241c1cf8a144") : secret "insights-runtime-extractor-tls" not found
Sep 09 21:34:12.224765 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:12.224720    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm podName:e282a8be-3ba4-42d0-ade2-241c1cf8a144 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:13.224710832 +0000 UTC m=+72.697675740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mwjgp" (UID: "e282a8be-3ba4-42d0-ade2-241c1cf8a144") : configmap "kube-rbac-proxy" not found
Sep 09 21:34:12.285583 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:12.285555    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bdlgh" event={"ID":"885c3bcb-44b1-46c4-ab1c-62269c8d5474","Type":"ContainerStarted","Data":"7314f7db34546ed7d72f5c69af8f234651ce3ed1e39c551933d5cbfa60817484"}
Sep 09 21:34:12.285717 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:12.285661    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:34:12.287213 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:12.287198    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 21:34:12.287616 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:12.287601    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/0.log"
Sep 09 21:34:12.287682 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:12.287635    2578 generic.go:356] "Generic (PLEG): container finished" podID="14f44884-6479-48c1-9961-717810d01fc8" containerID="429cdc02b652b58430c31c306d75166410e012b553780c1fd982afa2b800212e" exitCode=255
Sep 09 21:34:12.287727 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:12.287694    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-mksnd" event={"ID":"14f44884-6479-48c1-9961-717810d01fc8","Type":"ContainerDied","Data":"429cdc02b652b58430c31c306d75166410e012b553780c1fd982afa2b800212e"}
Sep 09 21:34:12.287774 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:12.287735    2578 scope.go:117] "RemoveContainer" containerID="125f7ecdc7e71ff88a7b97a0d3a7e033c7797244e484a4c37675cb53ce38a407"
Sep 09 21:34:12.287907 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:12.287893    2578 scope.go:117] "RemoveContainer" containerID="429cdc02b652b58430c31c306d75166410e012b553780c1fd982afa2b800212e"
Sep 09 21:34:12.288098 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:12.288063    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-mksnd_openshift-console-operator(14f44884-6479-48c1-9961-717810d01fc8)\"" pod="openshift-console-operator/console-operator-85b454874b-mksnd" podUID="14f44884-6479-48c1-9961-717810d01fc8"
Sep 09 21:34:12.289261 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:12.289240    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-69ls5" event={"ID":"1995b548-a1df-4e1f-81ea-0f66bc72bf58","Type":"ContainerStarted","Data":"c7d18215080fb579c88a91da0f61c8354e5ddcf59fa146be1cba43796197ad36"}
Sep 09 21:34:12.299066 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:12.299025    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bdlgh" podStartSLOduration=65.379104461 podStartE2EDuration="1m10.299013739s" totalImagesPullingTime="4.919909278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:34:12.298550328 +0000 UTC m=+71.771515257" watchObservedRunningTime="2026-09-09 21:34:12.299013739 +0000 UTC m=+71.771978649"
Sep 09 21:34:12.327222 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:12.327188    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-69ls5" podStartSLOduration=65.3851235 podStartE2EDuration="1m10.32717694s" totalImagesPullingTime="4.94205344s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:34:12.32650662 +0000 UTC m=+71.799471534" watchObservedRunningTime="2026-09-09 21:34:12.32717694 +0000 UTC m=+71.800141846"
Sep 09 21:34:13.231396 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:13.231356    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:13.231836 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:13.231455    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:13.231836 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:13.231469    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 21:34:13.231836 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:13.231532    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm podName:e282a8be-3ba4-42d0-ade2-241c1cf8a144 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:15.231516193 +0000 UTC m=+74.704481081 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mwjgp" (UID: "e282a8be-3ba4-42d0-ade2-241c1cf8a144") : configmap "kube-rbac-proxy" not found
Sep 09 21:34:13.231836 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:13.231621    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 21:34:13.231836 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:13.231693    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls podName:e282a8be-3ba4-42d0-ade2-241c1cf8a144 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:15.23167474 +0000 UTC m=+74.704639647 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mwjgp" (UID: "e282a8be-3ba4-42d0-ade2-241c1cf8a144") : secret "insights-runtime-extractor-tls" not found
Sep 09 21:34:13.294224 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:13.294201    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 21:34:13.294652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:13.294633    2578 scope.go:117] "RemoveContainer" containerID="429cdc02b652b58430c31c306d75166410e012b553780c1fd982afa2b800212e"
Sep 09 21:34:13.294810 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:13.294789    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-mksnd_openshift-console-operator(14f44884-6479-48c1-9961-717810d01fc8)\"" pod="openshift-console-operator/console-operator-85b454874b-mksnd" podUID="14f44884-6479-48c1-9961-717810d01fc8"
Sep 09 21:34:13.332354 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:13.332326    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mb2cr\" (UID: \"ee4873ec-62e9-4389-bc83-7b2736c20d08\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr"
Sep 09 21:34:13.332488 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:13.332458    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 21:34:13.332552 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:13.332518    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert podName:ee4873ec-62e9-4389-bc83-7b2736c20d08 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:21.332498492 +0000 UTC m=+80.805463401 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mb2cr" (UID: "ee4873ec-62e9-4389-bc83-7b2736c20d08") : secret "networking-console-plugin-cert" not found
Sep 09 21:34:13.533321 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:13.533297    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:13.533426 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:13.533326    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:13.533426 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:13.533399    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 21:34:13.533496 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:13.533433    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle podName:ab94f25b-423a-42a4-8784-2004122e692d nodeName:}" failed. No retries permitted until 2026-09-09 21:34:21.533419029 +0000 UTC m=+81.006383922 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle") pod "router-default-7685746588-wcx7f" (UID: "ab94f25b-423a-42a4-8784-2004122e692d") : configmap references non-existent config key: service-ca.crt
Sep 09 21:34:13.533496 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:13.533455    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs podName:ab94f25b-423a-42a4-8784-2004122e692d nodeName:}" failed. No retries permitted until 2026-09-09 21:34:21.533447456 +0000 UTC m=+81.006412347 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs") pod "router-default-7685746588-wcx7f" (UID: "ab94f25b-423a-42a4-8784-2004122e692d") : secret "router-metrics-certs-default" not found
Sep 09 21:34:14.169704 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:14.169678    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jrklh_1a022781-ea0f-4915-96be-242aac7202e6/dns-node-resolver/0.log"
Sep 09 21:34:15.246759 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:15.246722    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:15.247150 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:15.246791    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:15.247150 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:15.246847    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 21:34:15.247150 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:15.246907    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm podName:e282a8be-3ba4-42d0-ade2-241c1cf8a144 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:19.246893135 +0000 UTC m=+78.719858036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mwjgp" (UID: "e282a8be-3ba4-42d0-ade2-241c1cf8a144") : configmap "kube-rbac-proxy" not found
Sep 09 21:34:15.247150 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:15.246936    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 21:34:15.247150 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:15.247005    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls podName:e282a8be-3ba4-42d0-ade2-241c1cf8a144 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:19.246992456 +0000 UTC m=+78.719957349 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mwjgp" (UID: "e282a8be-3ba4-42d0-ade2-241c1cf8a144") : secret "insights-runtime-extractor-tls" not found
Sep 09 21:34:15.569976 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:15.569952    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wcng5_a97e012d-3878-4dc9-a0c4-9f0dae4cd093/node-ca/0.log"
Sep 09 21:34:15.969408 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:15.969333    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-59n6x_21d909c2-ce09-479f-b353-8b23ce49954d/serve-healthcheck-canary/0.log"
Sep 09 21:34:16.132573 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:16.132541    2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:16.132889 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:16.132876    2578 scope.go:117] "RemoveContainer" containerID="429cdc02b652b58430c31c306d75166410e012b553780c1fd982afa2b800212e"
Sep 09 21:34:16.133058 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:16.133038    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-mksnd_openshift-console-operator(14f44884-6479-48c1-9961-717810d01fc8)\"" pod="openshift-console-operator/console-operator-85b454874b-mksnd" podUID="14f44884-6479-48c1-9961-717810d01fc8"
Sep 09 21:34:16.974244 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:16.971719    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-6nvrn_226b4308-cb75-45b3-aaa6-467856e4d5eb/kube-storage-version-migrator-operator/0.log"
Sep 09 21:34:19.280648 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:19.280609    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:19.281182 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:19.280671    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:19.281182 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:19.280720    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 21:34:19.281182 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:19.280754    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 21:34:19.281182 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:19.280795    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm podName:e282a8be-3ba4-42d0-ade2-241c1cf8a144 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:27.28077937 +0000 UTC m=+86.753744282 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mwjgp" (UID: "e282a8be-3ba4-42d0-ade2-241c1cf8a144") : configmap "kube-rbac-proxy" not found
Sep 09 21:34:19.281182 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:19.280811    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls podName:e282a8be-3ba4-42d0-ade2-241c1cf8a144 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:27.280803359 +0000 UTC m=+86.753768246 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mwjgp" (UID: "e282a8be-3ba4-42d0-ade2-241c1cf8a144") : secret "insights-runtime-extractor-tls" not found
Sep 09 21:34:20.274948 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:20.274895    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:20.275329 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:20.275314    2578 scope.go:117] "RemoveContainer" containerID="429cdc02b652b58430c31c306d75166410e012b553780c1fd982afa2b800212e"
Sep 09 21:34:20.275492 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:20.275472    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-mksnd_openshift-console-operator(14f44884-6479-48c1-9961-717810d01fc8)\"" pod="openshift-console-operator/console-operator-85b454874b-mksnd" podUID="14f44884-6479-48c1-9961-717810d01fc8"
Sep 09 21:34:21.395118 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:21.395084    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mb2cr\" (UID: \"ee4873ec-62e9-4389-bc83-7b2736c20d08\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr"
Sep 09 21:34:21.395492 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:21.395243    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 21:34:21.395492 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:21.395298    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert podName:ee4873ec-62e9-4389-bc83-7b2736c20d08 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:37.395284466 +0000 UTC m=+96.868249393 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mb2cr" (UID: "ee4873ec-62e9-4389-bc83-7b2736c20d08") : secret "networking-console-plugin-cert" not found
Sep 09 21:34:21.596849 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:21.596817    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:21.596849 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:21.596851    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:21.597490 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:21.597472    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab94f25b-423a-42a4-8784-2004122e692d-service-ca-bundle\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:21.599189 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:21.599171    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab94f25b-423a-42a4-8784-2004122e692d-metrics-certs\") pod \"router-default-7685746588-wcx7f\" (UID: \"ab94f25b-423a-42a4-8784-2004122e692d\") " pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:21.737300 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:21.737224    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:22.003184 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:22.003110    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7685746588-wcx7f"]
Sep 09 21:34:22.007103 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:22.007076    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab94f25b_423a_42a4_8784_2004122e692d.slice/crio-73a35fccc6712ce0ff605266e32d6ceba1576f7fa0fffb7bb6c98ac8a991b360 WatchSource:0}: Error finding container 73a35fccc6712ce0ff605266e32d6ceba1576f7fa0fffb7bb6c98ac8a991b360: Status 404 returned error can't find the container with id 73a35fccc6712ce0ff605266e32d6ceba1576f7fa0fffb7bb6c98ac8a991b360
Sep 09 21:34:22.321686 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:22.321655    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7685746588-wcx7f" event={"ID":"ab94f25b-423a-42a4-8784-2004122e692d","Type":"ContainerStarted","Data":"2032a0a1417997f724ea785a2b1c78866a18cfdd06d4447ec37c7e719b91108f"}
Sep 09 21:34:22.321840 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:22.321704    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7685746588-wcx7f" event={"ID":"ab94f25b-423a-42a4-8784-2004122e692d","Type":"ContainerStarted","Data":"73a35fccc6712ce0ff605266e32d6ceba1576f7fa0fffb7bb6c98ac8a991b360"}
Sep 09 21:34:22.341184 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:22.341134    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7685746588-wcx7f" podStartSLOduration=17.34111914 podStartE2EDuration="17.34111914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:34:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:34:22.340389126 +0000 UTC m=+81.813354035" watchObservedRunningTime="2026-09-09 21:34:22.34111914 +0000 UTC m=+81.814084049"
Sep 09 21:34:22.738457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:22.738388    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:22.741035 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:22.741008    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:23.324821 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:23.324790    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:23.325930 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:23.325911    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7685746588-wcx7f"
Sep 09 21:34:27.339986 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:27.339931    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:27.340372 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:27.340027    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:27.340563 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:27.340542    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e282a8be-3ba4-42d0-ade2-241c1cf8a144-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:27.342377 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:27.342357    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e282a8be-3ba4-42d0-ade2-241c1cf8a144-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mwjgp\" (UID: \"e282a8be-3ba4-42d0-ade2-241c1cf8a144\") " pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:27.509009 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:27.508957    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mwjgp"
Sep 09 21:34:27.779219 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:27.779193    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mwjgp"]
Sep 09 21:34:27.782034 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:27.782009    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode282a8be_3ba4_42d0_ade2_241c1cf8a144.slice/crio-30011f7ab06612de485475694998303c8734e324962a3914faf2e89e7a800c3c WatchSource:0}: Error finding container 30011f7ab06612de485475694998303c8734e324962a3914faf2e89e7a800c3c: Status 404 returned error can't find the container with id 30011f7ab06612de485475694998303c8734e324962a3914faf2e89e7a800c3c
Sep 09 21:34:28.341917 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:28.341887    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mwjgp" event={"ID":"e282a8be-3ba4-42d0-ade2-241c1cf8a144","Type":"ContainerStarted","Data":"838e8e4e9a38a311963ce803c2ba702f3599d18f7a55bd5fd380bb7848999f56"}
Sep 09 21:34:28.342285 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:28.341938    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mwjgp" event={"ID":"e282a8be-3ba4-42d0-ade2-241c1cf8a144","Type":"ContainerStarted","Data":"30011f7ab06612de485475694998303c8734e324962a3914faf2e89e7a800c3c"}
Sep 09 21:34:29.348695 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:29.348663    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mwjgp" event={"ID":"e282a8be-3ba4-42d0-ade2-241c1cf8a144","Type":"ContainerStarted","Data":"99620d1930781584ff4db0932975d88dd281cde9e08ac37033ff77f3cc16c026"}
Sep 09 21:34:31.359389 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:31.359353    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mwjgp" event={"ID":"e282a8be-3ba4-42d0-ade2-241c1cf8a144","Type":"ContainerStarted","Data":"b5e62f6544d2196e410e9f4dfb305bb38ae01291c3940094e530e4e00549e45e"}
Sep 09 21:34:31.375978 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:31.375929    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mwjgp" podStartSLOduration=17.8752867 podStartE2EDuration="20.375916353s" totalImagesPullingTime="2.500629653s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:34:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:34:31.374489056 +0000 UTC m=+90.847453964" watchObservedRunningTime="2026-09-09 21:34:31.375916353 +0000 UTC m=+90.848881264"
Sep 09 21:34:31.997312 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:31.997275    2578 scope.go:117] "RemoveContainer" containerID="429cdc02b652b58430c31c306d75166410e012b553780c1fd982afa2b800212e"
Sep 09 21:34:32.364132 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:32.364110    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 21:34:32.364575 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:32.364237    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-mksnd" event={"ID":"14f44884-6479-48c1-9961-717810d01fc8","Type":"ContainerStarted","Data":"d8ff018d2de4035d2c527b89295dd4c751d81f5cc334d64905608ab1551bde53"}
Sep 09 21:34:32.364653 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:32.364634    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:32.448725 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:32.448705    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-mksnd"
Sep 09 21:34:34.824889 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:34.824854    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"]
Sep 09 21:34:34.827013 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:34.826990    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"
Sep 09 21:34:34.830233 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:34.830213    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 09 21:34:34.830364 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:34.830231    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 09 21:34:34.830364 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:34.830286    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 09 21:34:34.830364 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:34.830300    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 09 21:34:34.842762 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:34.842738    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"]
Sep 09 21:34:34.898314 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:34.898289    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sdb7\" (UniqueName: \"kubernetes.io/projected/84bae543-be62-4fa8-8ea3-3e883b5feb24-kube-api-access-5sdb7\") pod \"klusterlet-addon-workmgr-5675fb9d48-bj49b\" (UID: \"84bae543-be62-4fa8-8ea3-3e883b5feb24\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"
Sep 09 21:34:34.898453 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:34.898356    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/84bae543-be62-4fa8-8ea3-3e883b5feb24-klusterlet-config\") pod \"klusterlet-addon-workmgr-5675fb9d48-bj49b\" (UID: \"84bae543-be62-4fa8-8ea3-3e883b5feb24\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"
Sep 09 21:34:34.898453 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:34.898416    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84bae543-be62-4fa8-8ea3-3e883b5feb24-tmp\") pod \"klusterlet-addon-workmgr-5675fb9d48-bj49b\" (UID: \"84bae543-be62-4fa8-8ea3-3e883b5feb24\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"
Sep 09 21:34:34.999100 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:34.999072    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sdb7\" (UniqueName: \"kubernetes.io/projected/84bae543-be62-4fa8-8ea3-3e883b5feb24-kube-api-access-5sdb7\") pod \"klusterlet-addon-workmgr-5675fb9d48-bj49b\" (UID: \"84bae543-be62-4fa8-8ea3-3e883b5feb24\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"
Sep 09 21:34:34.999270 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:34.999125    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/84bae543-be62-4fa8-8ea3-3e883b5feb24-klusterlet-config\") pod \"klusterlet-addon-workmgr-5675fb9d48-bj49b\" (UID: \"84bae543-be62-4fa8-8ea3-3e883b5feb24\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"
Sep 09 21:34:34.999328 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:34.999280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84bae543-be62-4fa8-8ea3-3e883b5feb24-tmp\") pod \"klusterlet-addon-workmgr-5675fb9d48-bj49b\" (UID: \"84bae543-be62-4fa8-8ea3-3e883b5feb24\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"
Sep 09 21:34:34.999740 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:34.999713    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84bae543-be62-4fa8-8ea3-3e883b5feb24-tmp\") pod \"klusterlet-addon-workmgr-5675fb9d48-bj49b\" (UID: \"84bae543-be62-4fa8-8ea3-3e883b5feb24\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"
Sep 09 21:34:35.002075 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:35.002052    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/84bae543-be62-4fa8-8ea3-3e883b5feb24-klusterlet-config\") pod \"klusterlet-addon-workmgr-5675fb9d48-bj49b\" (UID: \"84bae543-be62-4fa8-8ea3-3e883b5feb24\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"
Sep 09 21:34:35.007852 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:35.007831    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sdb7\" (UniqueName: \"kubernetes.io/projected/84bae543-be62-4fa8-8ea3-3e883b5feb24-kube-api-access-5sdb7\") pod \"klusterlet-addon-workmgr-5675fb9d48-bj49b\" (UID: \"84bae543-be62-4fa8-8ea3-3e883b5feb24\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"
Sep 09 21:34:35.138288 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:35.138211    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"
Sep 09 21:34:35.400718 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:35.400654    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"]
Sep 09 21:34:35.403533 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:35.403506    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84bae543_be62_4fa8_8ea3_3e883b5feb24.slice/crio-cb7274613bf9fae6b7359d889754e2a13a71f44898a268209c156131abefe2ec WatchSource:0}: Error finding container cb7274613bf9fae6b7359d889754e2a13a71f44898a268209c156131abefe2ec: Status 404 returned error can't find the container with id cb7274613bf9fae6b7359d889754e2a13a71f44898a268209c156131abefe2ec
Sep 09 21:34:36.379698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:36.379630    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b" event={"ID":"84bae543-be62-4fa8-8ea3-3e883b5feb24","Type":"ContainerStarted","Data":"cb7274613bf9fae6b7359d889754e2a13a71f44898a268209c156131abefe2ec"}
Sep 09 21:34:37.417866 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:37.417824    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mb2cr\" (UID: \"ee4873ec-62e9-4389-bc83-7b2736c20d08\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr"
Sep 09 21:34:37.420751 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:37.420696    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ee4873ec-62e9-4389-bc83-7b2736c20d08-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mb2cr\" (UID: \"ee4873ec-62e9-4389-bc83-7b2736c20d08\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr"
Sep 09 21:34:37.423537 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:37.423514    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr"
Sep 09 21:34:37.967140 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:37.967071    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-mb2cr"]
Sep 09 21:34:38.731378 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:38.731295    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:34:38.731378 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:38.731348    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:34:38.734177 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:38.734149    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/849ad12b-744d-4efa-ace8-0ba4dec756f1-metrics-tls\") pod \"dns-default-lcmzv\" (UID: \"849ad12b-744d-4efa-ace8-0ba4dec756f1\") " pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:34:38.734312 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:38.734183    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls\") pod \"image-registry-7858557fbc-hnwxm\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") " pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:34:38.766423 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:38.766395    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ts5sm\""
Sep 09 21:34:38.774247 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:38.774215    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:34:38.774363 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:38.774320    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k4v4f\""
Sep 09 21:34:38.782812 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:38.782790    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:34:39.085559 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:39.085522    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee4873ec_62e9_4389_bc83_7b2736c20d08.slice/crio-bc45e18e67d3648d447b9c743ab777ed0a44d7456928db0a9a1f7ed923905e57 WatchSource:0}: Error finding container bc45e18e67d3648d447b9c743ab777ed0a44d7456928db0a9a1f7ed923905e57: Status 404 returned error can't find the container with id bc45e18e67d3648d447b9c743ab777ed0a44d7456928db0a9a1f7ed923905e57
Sep 09 21:34:39.360331 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:39.360257    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lcmzv"]
Sep 09 21:34:39.363707 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:39.363681    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7858557fbc-hnwxm"]
Sep 09 21:34:39.363808 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:39.363725    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod849ad12b_744d_4efa_ace8_0ba4dec756f1.slice/crio-4d8d5e0ba399173b151d8d8eeba4f780732b7c03e2dc6a943e3b7e362feabea4 WatchSource:0}: Error finding container 4d8d5e0ba399173b151d8d8eeba4f780732b7c03e2dc6a943e3b7e362feabea4: Status 404 returned error can't find the container with id 4d8d5e0ba399173b151d8d8eeba4f780732b7c03e2dc6a943e3b7e362feabea4
Sep 09 21:34:39.366568 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:39.366550    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d6373bb_187e_4ec9_9e29_fff489dc7fd8.slice/crio-73023901fdaff20ffc3153c1d53b3df221ed92abdee63ac644928a5520dd779c WatchSource:0}: Error finding container 73023901fdaff20ffc3153c1d53b3df221ed92abdee63ac644928a5520dd779c: Status 404 returned error can't find the container with id 73023901fdaff20ffc3153c1d53b3df221ed92abdee63ac644928a5520dd779c
Sep 09 21:34:39.390701 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:39.390674    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b" event={"ID":"84bae543-be62-4fa8-8ea3-3e883b5feb24","Type":"ContainerStarted","Data":"762cb43056a5c67e37b1aa2074c84322e41763993879413ecbf06c8e22741680"}
Sep 09 21:34:39.390845 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:39.390828    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"
Sep 09 21:34:39.391881 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:39.391860    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lcmzv" event={"ID":"849ad12b-744d-4efa-ace8-0ba4dec756f1","Type":"ContainerStarted","Data":"4d8d5e0ba399173b151d8d8eeba4f780732b7c03e2dc6a943e3b7e362feabea4"}
Sep 09 21:34:39.392681 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:39.392659    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b"
Sep 09 21:34:39.393219 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:39.393189    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7858557fbc-hnwxm" event={"ID":"5d6373bb-187e-4ec9-9e29-fff489dc7fd8","Type":"ContainerStarted","Data":"73023901fdaff20ffc3153c1d53b3df221ed92abdee63ac644928a5520dd779c"}
Sep 09 21:34:39.394486 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:39.394469    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr" event={"ID":"ee4873ec-62e9-4389-bc83-7b2736c20d08","Type":"ContainerStarted","Data":"bc45e18e67d3648d447b9c743ab777ed0a44d7456928db0a9a1f7ed923905e57"}
Sep 09 21:34:39.410907 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:39.410867    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5675fb9d48-bj49b" podStartSLOduration=1.675164747 podStartE2EDuration="5.410854239s" totalImagesPullingTime="3.735689492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:34:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:34:39.40984748 +0000 UTC m=+98.882812389" watchObservedRunningTime="2026-09-09 21:34:39.410854239 +0000 UTC m=+98.883819148"
Sep 09 21:34:40.400022 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:40.399940    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7858557fbc-hnwxm" event={"ID":"5d6373bb-187e-4ec9-9e29-fff489dc7fd8","Type":"ContainerStarted","Data":"a7069e3ae8752e6808a6cf11071479a057a6f477c2ab135002d35a1bcb5c3a24"}
Sep 09 21:34:40.420188 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:40.420138    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7858557fbc-hnwxm" podStartSLOduration=98.420116868 podStartE2EDuration="1m38.420116868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:33:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:34:40.418768294 +0000 UTC m=+99.891733205" watchObservedRunningTime="2026-09-09 21:34:40.420116868 +0000 UTC m=+99.893081778"
Sep 09 21:34:41.405015 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:41.404959    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr" event={"ID":"ee4873ec-62e9-4389-bc83-7b2736c20d08","Type":"ContainerStarted","Data":"0d2dbaf0abc23af69e467b4b150784a2bbd1effaabad05f91afbbadaa1a92bab"}
Sep 09 21:34:41.405494 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:41.405291    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:34:41.419319 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:41.419272    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-mb2cr" podStartSLOduration=35.035198922 podStartE2EDuration="36.419257719s" totalImagesPullingTime="1.384058797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:34:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:34:41.418889615 +0000 UTC m=+100.891854524" watchObservedRunningTime="2026-09-09 21:34:41.419257719 +0000 UTC m=+100.892222628"
Sep 09 21:34:42.410302 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:42.410269    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lcmzv" event={"ID":"849ad12b-744d-4efa-ace8-0ba4dec756f1","Type":"ContainerStarted","Data":"a41da753fcc9c623706dfe81fb62a87bc83db74824948540eddec90a64781da8"}
Sep 09 21:34:42.410666 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:42.410326    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lcmzv" event={"ID":"849ad12b-744d-4efa-ace8-0ba4dec756f1","Type":"ContainerStarted","Data":"80a03f98270d73c9f806dabfd58eaa954e5fa6eaadf83ae65f3a8bdbb94b2847"}
Sep 09 21:34:42.427995 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:42.427935    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lcmzv" podStartSLOduration=66.288505229 podStartE2EDuration="1m8.427919685s" totalImagesPullingTime="2.139414456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:34:42.426754565 +0000 UTC m=+101.899719474" watchObservedRunningTime="2026-09-09 21:34:42.427919685 +0000 UTC m=+101.900884594"
Sep 09 21:34:42.785035 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:42.785006    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-fzxp6"]
Sep 09 21:34:42.799361 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:42.799334    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-fzxp6"]
Sep 09 21:34:42.799502 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:42.799373    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-fzxp6"
Sep 09 21:34:42.801639 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:42.801617    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 09 21:34:42.801748 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:42.801674    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-tsfkc\""
Sep 09 21:34:42.866036 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:42.866015    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/19be090a-b269-4250-9702-a60e5e6eaa20-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-fzxp6\" (UID: \"19be090a-b269-4250-9702-a60e5e6eaa20\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-fzxp6"
Sep 09 21:34:42.966491 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:42.966458    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/19be090a-b269-4250-9702-a60e5e6eaa20-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-fzxp6\" (UID: \"19be090a-b269-4250-9702-a60e5e6eaa20\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-fzxp6"
Sep 09 21:34:42.966655 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:42.966625    2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 09 21:34:42.966698 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:42.966685    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19be090a-b269-4250-9702-a60e5e6eaa20-tls-certificates podName:19be090a-b269-4250-9702-a60e5e6eaa20 nodeName:}" failed. No retries permitted until 2026-09-09 21:34:43.466668558 +0000 UTC m=+102.939633451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/19be090a-b269-4250-9702-a60e5e6eaa20-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-fzxp6" (UID: "19be090a-b269-4250-9702-a60e5e6eaa20") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 09 21:34:43.296561 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:43.296537    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bdlgh"
Sep 09 21:34:43.413776 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:43.413744    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:34:43.471244 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:43.471216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/19be090a-b269-4250-9702-a60e5e6eaa20-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-fzxp6\" (UID: \"19be090a-b269-4250-9702-a60e5e6eaa20\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-fzxp6"
Sep 09 21:34:43.473473 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:43.473454    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/19be090a-b269-4250-9702-a60e5e6eaa20-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-fzxp6\" (UID: \"19be090a-b269-4250-9702-a60e5e6eaa20\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-fzxp6"
Sep 09 21:34:43.708684 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:43.708616    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-fzxp6"
Sep 09 21:34:43.992149 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:43.992088    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-fzxp6"]
Sep 09 21:34:43.995155 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:43.995132    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19be090a_b269_4250_9702_a60e5e6eaa20.slice/crio-f83bfe0d92c686d30861fcff922f0bcacdfc37407f094d1e7710591e4346fc2a WatchSource:0}: Error finding container f83bfe0d92c686d30861fcff922f0bcacdfc37407f094d1e7710591e4346fc2a: Status 404 returned error can't find the container with id f83bfe0d92c686d30861fcff922f0bcacdfc37407f094d1e7710591e4346fc2a
Sep 09 21:34:44.417805 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:44.417771    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-fzxp6" event={"ID":"19be090a-b269-4250-9702-a60e5e6eaa20","Type":"ContainerStarted","Data":"f83bfe0d92c686d30861fcff922f0bcacdfc37407f094d1e7710591e4346fc2a"}
Sep 09 21:34:47.429385 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:47.429350    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-fzxp6" event={"ID":"19be090a-b269-4250-9702-a60e5e6eaa20","Type":"ContainerStarted","Data":"81d74e35b82593a62659946c82cbb83db46a87921f6dbddf4ccc4961153dfcca"}
Sep 09 21:34:47.429813 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:47.429556    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-fzxp6"
Sep 09 21:34:47.434322 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:47.434296    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-fzxp6"
Sep 09 21:34:47.442644 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:47.442598    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-fzxp6" podStartSLOduration=2.653876211 podStartE2EDuration="5.442584637s" totalImagesPullingTime="2.788708426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:34:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:34:47.442283172 +0000 UTC m=+106.915248082" watchObservedRunningTime="2026-09-09 21:34:47.442584637 +0000 UTC m=+106.915549546"
Sep 09 21:34:47.840099 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:47.840068    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-ccczb"]
Sep 09 21:34:47.842104 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:47.842088    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb"
Sep 09 21:34:47.844295 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:47.844275    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 09 21:34:47.845075 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:47.845051    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 21:34:47.845075 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:47.845061    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 09 21:34:47.845075 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:47.845072    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 21:34:47.845276 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:47.845065    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-98cmk\""
Sep 09 21:34:47.845509 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:47.845488    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 21:34:47.850485 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:47.850467    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-ccczb"]
Sep 09 21:34:48.001819 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:48.001780    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7cqg\" (UniqueName: \"kubernetes.io/projected/fef07e24-eca1-47da-b1ae-5adb2bc040ac-kube-api-access-x7cqg\") pod \"prometheus-operator-6f4946db56-ccczb\" (UID: \"fef07e24-eca1-47da-b1ae-5adb2bc040ac\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb"
Sep 09 21:34:48.001962 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:48.001828    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fef07e24-eca1-47da-b1ae-5adb2bc040ac-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ccczb\" (UID: \"fef07e24-eca1-47da-b1ae-5adb2bc040ac\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb"
Sep 09 21:34:48.001962 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:48.001848    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fef07e24-eca1-47da-b1ae-5adb2bc040ac-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-ccczb\" (UID: \"fef07e24-eca1-47da-b1ae-5adb2bc040ac\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb"
Sep 09 21:34:48.001962 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:48.001932    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fef07e24-eca1-47da-b1ae-5adb2bc040ac-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-ccczb\" (UID: \"fef07e24-eca1-47da-b1ae-5adb2bc040ac\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb"
Sep 09 21:34:48.102242 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:48.102182    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7cqg\" (UniqueName: \"kubernetes.io/projected/fef07e24-eca1-47da-b1ae-5adb2bc040ac-kube-api-access-x7cqg\") pod \"prometheus-operator-6f4946db56-ccczb\" (UID: \"fef07e24-eca1-47da-b1ae-5adb2bc040ac\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb"
Sep 09 21:34:48.102242 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:48.102217    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fef07e24-eca1-47da-b1ae-5adb2bc040ac-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ccczb\" (UID: \"fef07e24-eca1-47da-b1ae-5adb2bc040ac\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb"
Sep 09 21:34:48.102242 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:48.102236    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fef07e24-eca1-47da-b1ae-5adb2bc040ac-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-ccczb\" (UID: \"fef07e24-eca1-47da-b1ae-5adb2bc040ac\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb"
Sep 09 21:34:48.102412 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:48.102285    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fef07e24-eca1-47da-b1ae-5adb2bc040ac-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-ccczb\" (UID: \"fef07e24-eca1-47da-b1ae-5adb2bc040ac\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb"
Sep 09 21:34:48.102412 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:48.102336    2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 09 21:34:48.102412 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:34:48.102393    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fef07e24-eca1-47da-b1ae-5adb2bc040ac-prometheus-operator-tls podName:fef07e24-eca1-47da-b1ae-5adb2bc040ac nodeName:}" failed. No retries permitted until 2026-09-09 21:34:48.6023784 +0000 UTC m=+108.075343288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/fef07e24-eca1-47da-b1ae-5adb2bc040ac-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-ccczb" (UID: "fef07e24-eca1-47da-b1ae-5adb2bc040ac") : secret "prometheus-operator-tls" not found
Sep 09 21:34:48.102988 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:48.102952    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fef07e24-eca1-47da-b1ae-5adb2bc040ac-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-ccczb\" (UID: \"fef07e24-eca1-47da-b1ae-5adb2bc040ac\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb"
Sep 09 21:34:48.104681 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:48.104659    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fef07e24-eca1-47da-b1ae-5adb2bc040ac-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-ccczb\" (UID: \"fef07e24-eca1-47da-b1ae-5adb2bc040ac\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb"
Sep 09 21:34:48.111019 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:48.110998    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7cqg\" (UniqueName: \"kubernetes.io/projected/fef07e24-eca1-47da-b1ae-5adb2bc040ac-kube-api-access-x7cqg\") pod \"prometheus-operator-6f4946db56-ccczb\" (UID: \"fef07e24-eca1-47da-b1ae-5adb2bc040ac\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb"
Sep 09 21:34:48.606191 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:48.606160    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fef07e24-eca1-47da-b1ae-5adb2bc040ac-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ccczb\" (UID: \"fef07e24-eca1-47da-b1ae-5adb2bc040ac\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb"
Sep 09 21:34:48.608554 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:48.608525    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fef07e24-eca1-47da-b1ae-5adb2bc040ac-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ccczb\" (UID: \"fef07e24-eca1-47da-b1ae-5adb2bc040ac\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb"
Sep 09 21:34:48.750788 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:48.750760    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb"
Sep 09 21:34:49.015541 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:49.015513    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-ccczb"]
Sep 09 21:34:49.018876 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:49.018850    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfef07e24_eca1_47da_b1ae_5adb2bc040ac.slice/crio-5efd858b58520f26a69271a4de317dfff8ce985f54e15c4651d5ba01fdd7b04c WatchSource:0}: Error finding container 5efd858b58520f26a69271a4de317dfff8ce985f54e15c4651d5ba01fdd7b04c: Status 404 returned error can't find the container with id 5efd858b58520f26a69271a4de317dfff8ce985f54e15c4651d5ba01fdd7b04c
Sep 09 21:34:49.437879 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:49.437800    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb" event={"ID":"fef07e24-eca1-47da-b1ae-5adb2bc040ac","Type":"ContainerStarted","Data":"5efd858b58520f26a69271a4de317dfff8ce985f54e15c4651d5ba01fdd7b04c"}
Sep 09 21:34:51.448011 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:51.447962    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb" event={"ID":"fef07e24-eca1-47da-b1ae-5adb2bc040ac","Type":"ContainerStarted","Data":"1539900b7acbebeaa10ea8e284394daa8668dfaba44f2e59480956d6d995dd20"}
Sep 09 21:34:51.448430 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:51.448033    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb" event={"ID":"fef07e24-eca1-47da-b1ae-5adb2bc040ac","Type":"ContainerStarted","Data":"654b0b2e87d86423670c471b840b33d39fcdc80f361fb0f7c4bec8c9ddb1df2a"}
Sep 09 21:34:51.464172 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:51.464125    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-ccczb" podStartSLOduration=3.035505537 podStartE2EDuration="4.464110135s" totalImagesPullingTime="1.428604598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:34:51.462476817 +0000 UTC m=+110.935441728" watchObservedRunningTime="2026-09-09 21:34:51.464110135 +0000 UTC m=+110.937075044"
Sep 09 21:34:53.201795 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.201761    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"]
Sep 09 21:34:53.203938 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.203918    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"
Sep 09 21:34:53.206052 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.206030    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bp6wd\""
Sep 09 21:34:53.206238 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.206221    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 09 21:34:53.206287 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.206260    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 09 21:34:53.214474 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.214451    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"]
Sep 09 21:34:53.229464 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.229446    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jkn9v"]
Sep 09 21:34:53.231527 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.231510    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.234068 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.234051    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 21:34:53.234323 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.234303    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 21:34:53.234435 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.234324    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9tb2f\""
Sep 09 21:34:53.234435 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.234348    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 21:34:53.340894 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.340862    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35e853f0-ec73-457e-9a6d-a5c4af34588f-node-exporter-accelerators-collector-config\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.340894 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.340893    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35e853f0-ec73-457e-9a6d-a5c4af34588f-sys\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.341100 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.340913    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0a6de4a-b76b-4eb2-9e9b-b1452d8881be-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-v2gwl\" (UID: \"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"
Sep 09 21:34:53.341100 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.340935    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k7sr\" (UniqueName: \"kubernetes.io/projected/f0a6de4a-b76b-4eb2-9e9b-b1452d8881be-kube-api-access-9k7sr\") pod \"openshift-state-metrics-657fd6c54-v2gwl\" (UID: \"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"
Sep 09 21:34:53.341100 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.340955    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35e853f0-ec73-457e-9a6d-a5c4af34588f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.341100 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.341021    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0a6de4a-b76b-4eb2-9e9b-b1452d8881be-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-v2gwl\" (UID: \"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"
Sep 09 21:34:53.341100 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.341080    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f0a6de4a-b76b-4eb2-9e9b-b1452d8881be-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-v2gwl\" (UID: \"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"
Sep 09 21:34:53.341252 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.341122    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35e853f0-ec73-457e-9a6d-a5c4af34588f-node-exporter-wtmp\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.341252 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.341143    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35e853f0-ec73-457e-9a6d-a5c4af34588f-root\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.341252 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.341196    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35e853f0-ec73-457e-9a6d-a5c4af34588f-node-exporter-tls\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.341252 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.341217    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35e853f0-ec73-457e-9a6d-a5c4af34588f-node-exporter-textfile\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.341252 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.341235    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35e853f0-ec73-457e-9a6d-a5c4af34588f-metrics-client-ca\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.341398 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.341281    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7czrl\" (UniqueName: \"kubernetes.io/projected/35e853f0-ec73-457e-9a6d-a5c4af34588f-kube-api-access-7czrl\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.420095 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.420073    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lcmzv"
Sep 09 21:34:53.442331 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442309    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35e853f0-ec73-457e-9a6d-a5c4af34588f-metrics-client-ca\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.442443 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442340    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7czrl\" (UniqueName: \"kubernetes.io/projected/35e853f0-ec73-457e-9a6d-a5c4af34588f-kube-api-access-7czrl\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.442443 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442358    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35e853f0-ec73-457e-9a6d-a5c4af34588f-node-exporter-accelerators-collector-config\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.442443 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442374    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35e853f0-ec73-457e-9a6d-a5c4af34588f-sys\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.442443 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442396    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0a6de4a-b76b-4eb2-9e9b-b1452d8881be-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-v2gwl\" (UID: \"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"
Sep 09 21:34:53.442443 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442427    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k7sr\" (UniqueName: \"kubernetes.io/projected/f0a6de4a-b76b-4eb2-9e9b-b1452d8881be-kube-api-access-9k7sr\") pod \"openshift-state-metrics-657fd6c54-v2gwl\" (UID: \"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"
Sep 09 21:34:53.442683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442446    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35e853f0-ec73-457e-9a6d-a5c4af34588f-sys\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.442683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442452    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35e853f0-ec73-457e-9a6d-a5c4af34588f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.442683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442496    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0a6de4a-b76b-4eb2-9e9b-b1452d8881be-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-v2gwl\" (UID: \"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"
Sep 09 21:34:53.442683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442542    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f0a6de4a-b76b-4eb2-9e9b-b1452d8881be-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-v2gwl\" (UID: \"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"
Sep 09 21:34:53.442683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442586    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35e853f0-ec73-457e-9a6d-a5c4af34588f-node-exporter-wtmp\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.442683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442611    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35e853f0-ec73-457e-9a6d-a5c4af34588f-root\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.442683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442645    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35e853f0-ec73-457e-9a6d-a5c4af34588f-node-exporter-tls\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.442683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442673    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35e853f0-ec73-457e-9a6d-a5c4af34588f-node-exporter-textfile\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.443124 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442838    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35e853f0-ec73-457e-9a6d-a5c4af34588f-root\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.443124 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442946    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35e853f0-ec73-457e-9a6d-a5c4af34588f-node-exporter-wtmp\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.443124 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.442961    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35e853f0-ec73-457e-9a6d-a5c4af34588f-node-exporter-accelerators-collector-config\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.443124 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.443022    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35e853f0-ec73-457e-9a6d-a5c4af34588f-metrics-client-ca\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.443124 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.443078    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35e853f0-ec73-457e-9a6d-a5c4af34588f-node-exporter-textfile\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.443363 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.443279    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0a6de4a-b76b-4eb2-9e9b-b1452d8881be-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-v2gwl\" (UID: \"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"
Sep 09 21:34:53.445406 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.445379    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35e853f0-ec73-457e-9a6d-a5c4af34588f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.445561 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.445536    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f0a6de4a-b76b-4eb2-9e9b-b1452d8881be-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-v2gwl\" (UID: \"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"
Sep 09 21:34:53.445686 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.445564    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35e853f0-ec73-457e-9a6d-a5c4af34588f-node-exporter-tls\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.445766 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.445744    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f0a6de4a-b76b-4eb2-9e9b-b1452d8881be-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-v2gwl\" (UID: \"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"
Sep 09 21:34:53.449841 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.449822    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k7sr\" (UniqueName: \"kubernetes.io/projected/f0a6de4a-b76b-4eb2-9e9b-b1452d8881be-kube-api-access-9k7sr\") pod \"openshift-state-metrics-657fd6c54-v2gwl\" (UID: \"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"
Sep 09 21:34:53.449936 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.449853    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7czrl\" (UniqueName: \"kubernetes.io/projected/35e853f0-ec73-457e-9a6d-a5c4af34588f-kube-api-access-7czrl\") pod \"node-exporter-jkn9v\" (UID: \"35e853f0-ec73-457e-9a6d-a5c4af34588f\") " pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.513464 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.513409    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"
Sep 09 21:34:53.539948 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.539923    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jkn9v"
Sep 09 21:34:53.548760 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:53.548736    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e853f0_ec73_457e_9a6d_a5c4af34588f.slice/crio-172620947dd71474b12674bb4f0d2398334ce36992641e83412aed5f32e45613 WatchSource:0}: Error finding container 172620947dd71474b12674bb4f0d2398334ce36992641e83412aed5f32e45613: Status 404 returned error can't find the container with id 172620947dd71474b12674bb4f0d2398334ce36992641e83412aed5f32e45613
Sep 09 21:34:53.785490 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:53.785466    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl"]
Sep 09 21:34:53.787686 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:53.787662    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a6de4a_b76b_4eb2_9e9b_b1452d8881be.slice/crio-33701cced505d20aa77bcca7ecd80367a776f4296268449fefa8364185344610 WatchSource:0}: Error finding container 33701cced505d20aa77bcca7ecd80367a776f4296268449fefa8364185344610: Status 404 returned error can't find the container with id 33701cced505d20aa77bcca7ecd80367a776f4296268449fefa8364185344610
Sep 09 21:34:54.461209 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:54.461138    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkn9v" event={"ID":"35e853f0-ec73-457e-9a6d-a5c4af34588f","Type":"ContainerStarted","Data":"172620947dd71474b12674bb4f0d2398334ce36992641e83412aed5f32e45613"}
Sep 09 21:34:54.464126 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:54.464094    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl" event={"ID":"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be","Type":"ContainerStarted","Data":"54d02b61555ec9fd388e5bbb9d4ffc6c3558d9c1d1da20e5ec2391da5e983c71"}
Sep 09 21:34:54.464253 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:54.464155    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl" event={"ID":"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be","Type":"ContainerStarted","Data":"080e98ee9ae0004c5118200bd2a071618d481ffa6888b8a9bc0394653195bc94"}
Sep 09 21:34:54.464253 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:54.464183    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl" event={"ID":"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be","Type":"ContainerStarted","Data":"33701cced505d20aa77bcca7ecd80367a776f4296268449fefa8364185344610"}
Sep 09 21:34:55.469749 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:55.469712    2578 generic.go:356] "Generic (PLEG): container finished" podID="35e853f0-ec73-457e-9a6d-a5c4af34588f" containerID="ce284db34df7f048e8d462bac6a3e618431528ac0cc80d23b9a0326919d6606f" exitCode=0
Sep 09 21:34:55.470222 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:55.469763    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkn9v" event={"ID":"35e853f0-ec73-457e-9a6d-a5c4af34588f","Type":"ContainerDied","Data":"ce284db34df7f048e8d462bac6a3e618431528ac0cc80d23b9a0326919d6606f"}
Sep 09 21:34:55.472218 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:55.472196    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl" event={"ID":"f0a6de4a-b76b-4eb2-9e9b-b1452d8881be","Type":"ContainerStarted","Data":"947ec379cfc148dc4c6c7ccc8b16c19d24319494b0ce043663179aed742d1b3e"}
Sep 09 21:34:55.504660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:55.504617    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-v2gwl" podStartSLOduration=1.385396279 podStartE2EDuration="2.50460008s" totalImagesPullingTime="1.119203801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:34:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:34:55.503605694 +0000 UTC m=+114.976570606" watchObservedRunningTime="2026-09-09 21:34:55.50460008 +0000 UTC m=+114.977564990"
Sep 09 21:34:56.478462 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:56.478421    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkn9v" event={"ID":"35e853f0-ec73-457e-9a6d-a5c4af34588f","Type":"ContainerStarted","Data":"703c861b8043d237e62c25d3dd23ee48b7885e8939210d535d4fcce0551f5573"}
Sep 09 21:34:56.478823 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:56.478517    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkn9v" event={"ID":"35e853f0-ec73-457e-9a6d-a5c4af34588f","Type":"ContainerStarted","Data":"a369c24429df4b9f8ce612ef293f7d97d77e456094c4cbf2bd54f37598b6f995"}
Sep 09 21:34:56.504069 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:56.504010    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jkn9v" podStartSLOduration=2.384791089 podStartE2EDuration="3.503964047s" totalImagesPullingTime="1.053086622s" totalInitContainerRuntime="66.086336ms" isStatefulPod=true podCreationTimestamp="2026-09-09 21:34:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:34:56.502666227 +0000 UTC m=+115.975631137" watchObservedRunningTime="2026-09-09 21:34:56.503964047 +0000 UTC m=+115.976928959"
Sep 09 21:34:57.647756 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.647727    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7f785858c4-m5bd7"]
Sep 09 21:34:57.650448 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.650429    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.653786 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.653756    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 09 21:34:57.653786 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.653756    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 09 21:34:57.653957 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.653814    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-e47d730bfbi8q\""
Sep 09 21:34:57.653957 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.653756    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 09 21:34:57.653957 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.653756    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 09 21:34:57.653957 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.653777    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-mvslw\""
Sep 09 21:34:57.659698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.659679    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f785858c4-m5bd7"]
Sep 09 21:34:57.779062 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.779029    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-metrics-server-audit-profiles\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.779062 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.779061    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.779274 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.779081    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-client-ca-bundle\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.779274 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.779102    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-secret-metrics-server-client-certs\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.779274 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.779157    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-audit-log\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.779274 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.779194    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzk7v\" (UniqueName: \"kubernetes.io/projected/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-kube-api-access-fzk7v\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.779274 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.779224    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-secret-metrics-server-tls\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.879711 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.879678    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-secret-metrics-server-client-certs\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.879711 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.879712    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-audit-log\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.879931 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.879735    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzk7v\" (UniqueName: \"kubernetes.io/projected/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-kube-api-access-fzk7v\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.879931 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.879757    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-secret-metrics-server-tls\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.879931 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.879812    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-metrics-server-audit-profiles\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.879931 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.879830    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.879931 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.879850    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-client-ca-bundle\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.880177 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.880155    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-audit-log\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.880747 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.880716    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.880867 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.880829    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-metrics-server-audit-profiles\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.882241 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.882217    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-secret-metrics-server-tls\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.882440 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.882418    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-client-ca-bundle\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.882521 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.882418    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-secret-metrics-server-client-certs\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.887034 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.887013    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzk7v\" (UniqueName: \"kubernetes.io/projected/cb4d3ab4-c14d-4698-909e-1cd3bc466e58-kube-api-access-fzk7v\") pod \"metrics-server-7f785858c4-m5bd7\" (UID: \"cb4d3ab4-c14d-4698-909e-1cd3bc466e58\") " pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.960068 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.959990    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:34:57.998345 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:57.998317    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p"]
Sep 09 21:34:58.000785 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.000765    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p"
Sep 09 21:34:58.002932 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.002913    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-gzm2f\""
Sep 09 21:34:58.002932 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.002926    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 09 21:34:58.007748 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.007692    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p"]
Sep 09 21:34:58.182333 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.182294    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b3d4319-b5e0-4f29-a0e4-fa4669144864-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-2588p\" (UID: \"9b3d4319-b5e0-4f29-a0e4-fa4669144864\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p"
Sep 09 21:34:58.182466 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.182348    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4hnf\" (UniqueName: \"kubernetes.io/projected/9b3d4319-b5e0-4f29-a0e4-fa4669144864-kube-api-access-m4hnf\") pod \"monitoring-plugin-85bbbbbf4-2588p\" (UID: \"9b3d4319-b5e0-4f29-a0e4-fa4669144864\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p"
Sep 09 21:34:58.231987 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.231913    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f785858c4-m5bd7"]
Sep 09 21:34:58.234631 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:58.234604    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb4d3ab4_c14d_4698_909e_1cd3bc466e58.slice/crio-1e68b867b59258dd6ee22ea98d2bd0595c6095dbd46acf4155ae01e68c79841b WatchSource:0}: Error finding container 1e68b867b59258dd6ee22ea98d2bd0595c6095dbd46acf4155ae01e68c79841b: Status 404 returned error can't find the container with id 1e68b867b59258dd6ee22ea98d2bd0595c6095dbd46acf4155ae01e68c79841b
Sep 09 21:34:58.283541 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.283516    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b3d4319-b5e0-4f29-a0e4-fa4669144864-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-2588p\" (UID: \"9b3d4319-b5e0-4f29-a0e4-fa4669144864\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p"
Sep 09 21:34:58.283624 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.283556    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4hnf\" (UniqueName: \"kubernetes.io/projected/9b3d4319-b5e0-4f29-a0e4-fa4669144864-kube-api-access-m4hnf\") pod \"monitoring-plugin-85bbbbbf4-2588p\" (UID: \"9b3d4319-b5e0-4f29-a0e4-fa4669144864\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p"
Sep 09 21:34:58.285671 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.285651    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9b3d4319-b5e0-4f29-a0e4-fa4669144864-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-2588p\" (UID: \"9b3d4319-b5e0-4f29-a0e4-fa4669144864\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p"
Sep 09 21:34:58.290429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.290405    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4hnf\" (UniqueName: \"kubernetes.io/projected/9b3d4319-b5e0-4f29-a0e4-fa4669144864-kube-api-access-m4hnf\") pod \"monitoring-plugin-85bbbbbf4-2588p\" (UID: \"9b3d4319-b5e0-4f29-a0e4-fa4669144864\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p"
Sep 09 21:34:58.315835 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.315816    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p"
Sep 09 21:34:58.486162 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.486087    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7" event={"ID":"cb4d3ab4-c14d-4698-909e-1cd3bc466e58","Type":"ContainerStarted","Data":"1e68b867b59258dd6ee22ea98d2bd0595c6095dbd46acf4155ae01e68c79841b"}
Sep 09 21:34:58.533405 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.532603    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7858557fbc-hnwxm"]
Sep 09 21:34:58.539247 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.539222    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:34:58.581865 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:58.581843    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p"]
Sep 09 21:34:58.584696 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:34:58.584675    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b3d4319_b5e0_4f29_a0e4_fa4669144864.slice/crio-b4e2144d4ccc83f8a62325ecd258ccb88cd37b9d659a02cd041d22cdeaad0641 WatchSource:0}: Error finding container b4e2144d4ccc83f8a62325ecd258ccb88cd37b9d659a02cd041d22cdeaad0641: Status 404 returned error can't find the container with id b4e2144d4ccc83f8a62325ecd258ccb88cd37b9d659a02cd041d22cdeaad0641
Sep 09 21:34:59.491515 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:34:59.491436    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p" event={"ID":"9b3d4319-b5e0-4f29-a0e4-fa4669144864","Type":"ContainerStarted","Data":"b4e2144d4ccc83f8a62325ecd258ccb88cd37b9d659a02cd041d22cdeaad0641"}
Sep 09 21:35:01.501168 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:01.501066    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p" event={"ID":"9b3d4319-b5e0-4f29-a0e4-fa4669144864","Type":"ContainerStarted","Data":"1451d003f29093bfbcd51720f25605c34c393b5674a1cbf009669b819ed168c8"}
Sep 09 21:35:01.501168 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:01.501165    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p"
Sep 09 21:35:01.502922 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:01.502881    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7" event={"ID":"cb4d3ab4-c14d-4698-909e-1cd3bc466e58","Type":"ContainerStarted","Data":"83cefc97a8f70528cea96f568f9eb3ecdeed1dd1085c31df7f6946031c97826d"}
Sep 09 21:35:01.506360 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:01.506336    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p"
Sep 09 21:35:01.516341 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:01.516298    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2588p" podStartSLOduration=2.076104206 podStartE2EDuration="4.516283398s" totalImagesPullingTime="2.440179192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:34:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:35:01.514718915 +0000 UTC m=+120.987683824" watchObservedRunningTime="2026-09-09 21:35:01.516283398 +0000 UTC m=+120.989248353"
Sep 09 21:35:01.530345 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:01.530215    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7" podStartSLOduration=1.7382859069999999 podStartE2EDuration="4.530199588s" totalImagesPullingTime="2.791913681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:34:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:35:01.529366867 +0000 UTC m=+121.002331776" watchObservedRunningTime="2026-09-09 21:35:01.530199588 +0000 UTC m=+121.003164498"
Sep 09 21:35:10.880321 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:10.880282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs\") pod \"network-metrics-daemon-sgvmj\" (UID: \"7c340b83-fda4-4fb8-afdd-9ec6d597ec62\") " pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:35:10.882565 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:10.882545    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c340b83-fda4-4fb8-afdd-9ec6d597ec62-metrics-certs\") pod \"network-metrics-daemon-sgvmj\" (UID: \"7c340b83-fda4-4fb8-afdd-9ec6d597ec62\") " pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:35:11.023873 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:11.023846    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5t55x\""
Sep 09 21:35:11.032584 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:11.032565    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sgvmj"
Sep 09 21:35:11.301139 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:11.301080    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sgvmj"]
Sep 09 21:35:11.303533 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:35:11.303502    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c340b83_fda4_4fb8_afdd_9ec6d597ec62.slice/crio-3e501e42471a340ded7f137516d986a1324b9e91ac489b67ac347f20ea1e6a0b WatchSource:0}: Error finding container 3e501e42471a340ded7f137516d986a1324b9e91ac489b67ac347f20ea1e6a0b: Status 404 returned error can't find the container with id 3e501e42471a340ded7f137516d986a1324b9e91ac489b67ac347f20ea1e6a0b
Sep 09 21:35:11.538359 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:11.538328    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sgvmj" event={"ID":"7c340b83-fda4-4fb8-afdd-9ec6d597ec62","Type":"ContainerStarted","Data":"3e501e42471a340ded7f137516d986a1324b9e91ac489b67ac347f20ea1e6a0b"}
Sep 09 21:35:12.543377 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:12.543333    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sgvmj" event={"ID":"7c340b83-fda4-4fb8-afdd-9ec6d597ec62","Type":"ContainerStarted","Data":"17e12d69d20ebe4b61912b3ad46ee491f968f45772edb68e83f1e00e7eb1847d"}
Sep 09 21:35:13.548563 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:13.548530    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sgvmj" event={"ID":"7c340b83-fda4-4fb8-afdd-9ec6d597ec62","Type":"ContainerStarted","Data":"eb5b55543560a3d9775c703a844bebcfc8194bec3103b1a01470f94597475ffc"}
Sep 09 21:35:13.564059 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:13.564013    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sgvmj" podStartSLOduration=130.487662298 podStartE2EDuration="2m11.563989163s" totalImagesPullingTime="1.076326865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:35:13.563067351 +0000 UTC m=+133.036032261" watchObservedRunningTime="2026-09-09 21:35:13.563989163 +0000 UTC m=+133.036954071"
Sep 09 21:35:17.960808 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:17.960772    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:35:17.961220 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:17.960858    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:35:23.552671 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.552629    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7858557fbc-hnwxm" podUID="5d6373bb-187e-4ec9-9e29-fff489dc7fd8" containerName="registry" containerID="cri-o://a7069e3ae8752e6808a6cf11071479a057a6f477c2ab135002d35a1bcb5c3a24" gracePeriod=30
Sep 09 21:35:23.795820 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.795799    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:35:23.869548 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.869493    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-certificates\") pod \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") "
Sep 09 21:35:23.869548 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.869531    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-bound-sa-token\") pod \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") "
Sep 09 21:35:23.869712 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.869562    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls\") pod \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") "
Sep 09 21:35:23.869712 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.869585    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-image-registry-private-configuration\") pod \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") "
Sep 09 21:35:23.869712 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.869617    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-ca-trust-extracted\") pod \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") "
Sep 09 21:35:23.869712 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.869653    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-trusted-ca\") pod \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") "
Sep 09 21:35:23.869712 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.869669    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-installation-pull-secrets\") pod \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") "
Sep 09 21:35:23.869948 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.869716    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvkfm\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-kube-api-access-fvkfm\") pod \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\" (UID: \"5d6373bb-187e-4ec9-9e29-fff489dc7fd8\") "
Sep 09 21:35:23.870281 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.870079    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5d6373bb-187e-4ec9-9e29-fff489dc7fd8" (UID: "5d6373bb-187e-4ec9-9e29-fff489dc7fd8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:35:23.870281 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.870242    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5d6373bb-187e-4ec9-9e29-fff489dc7fd8" (UID: "5d6373bb-187e-4ec9-9e29-fff489dc7fd8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:35:23.872421 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.872395    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-kube-api-access-fvkfm" (OuterVolumeSpecName: "kube-api-access-fvkfm") pod "5d6373bb-187e-4ec9-9e29-fff489dc7fd8" (UID: "5d6373bb-187e-4ec9-9e29-fff489dc7fd8"). InnerVolumeSpecName "kube-api-access-fvkfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:35:23.872512 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.872407    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5d6373bb-187e-4ec9-9e29-fff489dc7fd8" (UID: "5d6373bb-187e-4ec9-9e29-fff489dc7fd8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:35:23.872512 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.872473    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5d6373bb-187e-4ec9-9e29-fff489dc7fd8" (UID: "5d6373bb-187e-4ec9-9e29-fff489dc7fd8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:35:23.872576 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.872521    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5d6373bb-187e-4ec9-9e29-fff489dc7fd8" (UID: "5d6373bb-187e-4ec9-9e29-fff489dc7fd8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:35:23.872576 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.872551    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5d6373bb-187e-4ec9-9e29-fff489dc7fd8" (UID: "5d6373bb-187e-4ec9-9e29-fff489dc7fd8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:35:23.878092 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.878065    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5d6373bb-187e-4ec9-9e29-fff489dc7fd8" (UID: "5d6373bb-187e-4ec9-9e29-fff489dc7fd8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:35:23.971043 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.971011    2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-image-registry-private-configuration\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:35:23.971043 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.971042    2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-ca-trust-extracted\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:35:23.971043 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.971053    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-trusted-ca\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:35:23.971196 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.971061    2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-installation-pull-secrets\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:35:23.971196 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.971070    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvkfm\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-kube-api-access-fvkfm\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:35:23.971196 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.971079    2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-certificates\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:35:23.971196 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.971086    2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-bound-sa-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:35:23.971196 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:23.971094    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d6373bb-187e-4ec9-9e29-fff489dc7fd8-registry-tls\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:35:24.590069 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:24.590033    2578 generic.go:356] "Generic (PLEG): container finished" podID="5d6373bb-187e-4ec9-9e29-fff489dc7fd8" containerID="a7069e3ae8752e6808a6cf11071479a057a6f477c2ab135002d35a1bcb5c3a24" exitCode=0
Sep 09 21:35:24.590455 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:24.590074    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7858557fbc-hnwxm"
Sep 09 21:35:24.590455 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:24.590077    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7858557fbc-hnwxm" event={"ID":"5d6373bb-187e-4ec9-9e29-fff489dc7fd8","Type":"ContainerDied","Data":"a7069e3ae8752e6808a6cf11071479a057a6f477c2ab135002d35a1bcb5c3a24"}
Sep 09 21:35:24.590455 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:24.590120    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7858557fbc-hnwxm" event={"ID":"5d6373bb-187e-4ec9-9e29-fff489dc7fd8","Type":"ContainerDied","Data":"73023901fdaff20ffc3153c1d53b3df221ed92abdee63ac644928a5520dd779c"}
Sep 09 21:35:24.590455 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:24.590138    2578 scope.go:117] "RemoveContainer" containerID="a7069e3ae8752e6808a6cf11071479a057a6f477c2ab135002d35a1bcb5c3a24"
Sep 09 21:35:24.598493 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:24.598479    2578 scope.go:117] "RemoveContainer" containerID="a7069e3ae8752e6808a6cf11071479a057a6f477c2ab135002d35a1bcb5c3a24"
Sep 09 21:35:24.598745 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:35:24.598728    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7069e3ae8752e6808a6cf11071479a057a6f477c2ab135002d35a1bcb5c3a24\": container with ID starting with a7069e3ae8752e6808a6cf11071479a057a6f477c2ab135002d35a1bcb5c3a24 not found: ID does not exist" containerID="a7069e3ae8752e6808a6cf11071479a057a6f477c2ab135002d35a1bcb5c3a24"
Sep 09 21:35:24.598782 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:24.598753    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7069e3ae8752e6808a6cf11071479a057a6f477c2ab135002d35a1bcb5c3a24"} err="failed to get container status \"a7069e3ae8752e6808a6cf11071479a057a6f477c2ab135002d35a1bcb5c3a24\": rpc error: code = NotFound desc = could not find container \"a7069e3ae8752e6808a6cf11071479a057a6f477c2ab135002d35a1bcb5c3a24\": container with ID starting with a7069e3ae8752e6808a6cf11071479a057a6f477c2ab135002d35a1bcb5c3a24 not found: ID does not exist"
Sep 09 21:35:24.608652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:24.608628    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7858557fbc-hnwxm"]
Sep 09 21:35:24.612377 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:24.612358    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7858557fbc-hnwxm"]
Sep 09 21:35:25.001636 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:25.001582    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d6373bb-187e-4ec9-9e29-fff489dc7fd8" path="/var/lib/kubelet/pods/5d6373bb-187e-4ec9-9e29-fff489dc7fd8/volumes"
Sep 09 21:35:37.965798 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:37.965767    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:35:37.969472 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:37.969447    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7f785858c4-m5bd7"
Sep 09 21:35:40.651813 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:40.651779    2578 generic.go:356] "Generic (PLEG): container finished" podID="226b4308-cb75-45b3-aaa6-467856e4d5eb" containerID="21f103be882c898305a63a52f624cb51897ed0964b346d2cbd381921501037be" exitCode=0
Sep 09 21:35:40.652184 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:40.651854    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn" event={"ID":"226b4308-cb75-45b3-aaa6-467856e4d5eb","Type":"ContainerDied","Data":"21f103be882c898305a63a52f624cb51897ed0964b346d2cbd381921501037be"}
Sep 09 21:35:40.652184 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:40.652164    2578 scope.go:117] "RemoveContainer" containerID="21f103be882c898305a63a52f624cb51897ed0964b346d2cbd381921501037be"
Sep 09 21:35:41.656897 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:35:41.656864    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-6nvrn" event={"ID":"226b4308-cb75-45b3-aaa6-467856e4d5eb","Type":"ContainerStarted","Data":"318c28aa7870099ba9850424d0342fef6edc496d192356714d8c79efd46c8bd6"}
Sep 09 21:37:16.185679 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.185648    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"]
Sep 09 21:37:16.186138 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.185912    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d6373bb-187e-4ec9-9e29-fff489dc7fd8" containerName="registry"
Sep 09 21:37:16.186138 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.185922    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d6373bb-187e-4ec9-9e29-fff489dc7fd8" containerName="registry"
Sep 09 21:37:16.186138 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.185991    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d6373bb-187e-4ec9-9e29-fff489dc7fd8" containerName="registry"
Sep 09 21:37:16.189102 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.189074    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"
Sep 09 21:37:16.191408 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.191388    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9cxbc\""
Sep 09 21:37:16.191541 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.191416    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 21:37:16.192230 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.192213    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 21:37:16.197388 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.197366    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"]
Sep 09 21:37:16.253151 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.253131    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz\" (UID: \"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"
Sep 09 21:37:16.253253 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.253169    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr8xs\" (UniqueName: \"kubernetes.io/projected/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-kube-api-access-wr8xs\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz\" (UID: \"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"
Sep 09 21:37:16.253253 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.253206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz\" (UID: \"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"
Sep 09 21:37:16.353629 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.353606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr8xs\" (UniqueName: \"kubernetes.io/projected/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-kube-api-access-wr8xs\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz\" (UID: \"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"
Sep 09 21:37:16.353712 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.353645    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz\" (UID: \"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"
Sep 09 21:37:16.353712 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.353684    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz\" (UID: \"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"
Sep 09 21:37:16.354084 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.354065    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz\" (UID: \"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"
Sep 09 21:37:16.354129 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.354082    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz\" (UID: \"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"
Sep 09 21:37:16.361639 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.361611    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr8xs\" (UniqueName: \"kubernetes.io/projected/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-kube-api-access-wr8xs\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz\" (UID: \"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"
Sep 09 21:37:16.498877 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.498823    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"
Sep 09 21:37:16.767938 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:16.767800    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"]
Sep 09 21:37:16.770658 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:37:16.770630    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod272d27c8_c75e_4c7e_94b1_b3d7850fa1fc.slice/crio-c46a53afb241a76b966db90ca4b0a4c0166f5a881a38d5c077a9a48883c5656e WatchSource:0}: Error finding container c46a53afb241a76b966db90ca4b0a4c0166f5a881a38d5c077a9a48883c5656e: Status 404 returned error can't find the container with id c46a53afb241a76b966db90ca4b0a4c0166f5a881a38d5c077a9a48883c5656e
Sep 09 21:37:17.021234 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:17.021167    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz" event={"ID":"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc","Type":"ContainerStarted","Data":"c46a53afb241a76b966db90ca4b0a4c0166f5a881a38d5c077a9a48883c5656e"}
Sep 09 21:37:25.054533 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:25.054497    2578 generic.go:356] "Generic (PLEG): container finished" podID="272d27c8-c75e-4c7e-94b1-b3d7850fa1fc" containerID="b6e31bed83a15a47da51bfa110b5e5632c3597707063bdb63d4b4340edd23134" exitCode=0
Sep 09 21:37:25.054932 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:25.054544    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz" event={"ID":"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc","Type":"ContainerDied","Data":"b6e31bed83a15a47da51bfa110b5e5632c3597707063bdb63d4b4340edd23134"}
Sep 09 21:37:27.062834 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:27.062805    2578 generic.go:356] "Generic (PLEG): container finished" podID="272d27c8-c75e-4c7e-94b1-b3d7850fa1fc" containerID="61595aa9b39b5ff091191442445124c526a35b808a213b106c8eb6522020bd8c" exitCode=0
Sep 09 21:37:27.063135 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:27.062868    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz" event={"ID":"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc","Type":"ContainerDied","Data":"61595aa9b39b5ff091191442445124c526a35b808a213b106c8eb6522020bd8c"}
Sep 09 21:37:37.109847 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:37.109815    2578 generic.go:356] "Generic (PLEG): container finished" podID="272d27c8-c75e-4c7e-94b1-b3d7850fa1fc" containerID="982bc3a66858c3962951add8b423464a17d877ba473151ca828faac3fd2a338c" exitCode=0
Sep 09 21:37:37.110258 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:37.109909    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz" event={"ID":"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc","Type":"ContainerDied","Data":"982bc3a66858c3962951add8b423464a17d877ba473151ca828faac3fd2a338c"}
Sep 09 21:37:38.237196 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:38.237171    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"
Sep 09 21:37:38.323407 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:38.323382    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-bundle\") pod \"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc\" (UID: \"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc\") "
Sep 09 21:37:38.323560 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:38.323417    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-util\") pod \"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc\" (UID: \"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc\") "
Sep 09 21:37:38.323560 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:38.323449    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wr8xs\" (UniqueName: \"kubernetes.io/projected/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-kube-api-access-wr8xs\") pod \"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc\" (UID: \"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc\") "
Sep 09 21:37:38.324148 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:38.324120    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-bundle" (OuterVolumeSpecName: "bundle") pod "272d27c8-c75e-4c7e-94b1-b3d7850fa1fc" (UID: "272d27c8-c75e-4c7e-94b1-b3d7850fa1fc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:37:38.325620 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:38.325597    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-kube-api-access-wr8xs" (OuterVolumeSpecName: "kube-api-access-wr8xs") pod "272d27c8-c75e-4c7e-94b1-b3d7850fa1fc" (UID: "272d27c8-c75e-4c7e-94b1-b3d7850fa1fc"). InnerVolumeSpecName "kube-api-access-wr8xs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:37:38.327572 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:38.327551    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-util" (OuterVolumeSpecName: "util") pod "272d27c8-c75e-4c7e-94b1-b3d7850fa1fc" (UID: "272d27c8-c75e-4c7e-94b1-b3d7850fa1fc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:37:38.424855 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:38.424834    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-bundle\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:37:38.424855 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:38.424853    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-util\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:37:38.424995 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:38.424862    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wr8xs\" (UniqueName: \"kubernetes.io/projected/272d27c8-c75e-4c7e-94b1-b3d7850fa1fc-kube-api-access-wr8xs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:37:39.118632 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:39.118601    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz" event={"ID":"272d27c8-c75e-4c7e-94b1-b3d7850fa1fc","Type":"ContainerDied","Data":"c46a53afb241a76b966db90ca4b0a4c0166f5a881a38d5c077a9a48883c5656e"}
Sep 09 21:37:39.118751 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:39.118630    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407qqnz"
Sep 09 21:37:39.118751 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:39.118645    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c46a53afb241a76b966db90ca4b0a4c0166f5a881a38d5c077a9a48883c5656e"
Sep 09 21:37:43.998673 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:43.998638    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x7brk"]
Sep 09 21:37:43.999073 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:43.998898    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="272d27c8-c75e-4c7e-94b1-b3d7850fa1fc" containerName="pull"
Sep 09 21:37:43.999073 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:43.998909    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="272d27c8-c75e-4c7e-94b1-b3d7850fa1fc" containerName="pull"
Sep 09 21:37:43.999073 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:43.998922    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="272d27c8-c75e-4c7e-94b1-b3d7850fa1fc" containerName="util"
Sep 09 21:37:43.999073 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:43.998927    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="272d27c8-c75e-4c7e-94b1-b3d7850fa1fc" containerName="util"
Sep 09 21:37:43.999073 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:43.998937    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="272d27c8-c75e-4c7e-94b1-b3d7850fa1fc" containerName="extract"
Sep 09 21:37:43.999073 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:43.998943    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="272d27c8-c75e-4c7e-94b1-b3d7850fa1fc" containerName="extract"
Sep 09 21:37:43.999073 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:43.999012    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="272d27c8-c75e-4c7e-94b1-b3d7850fa1fc" containerName="extract"
Sep 09 21:37:44.002148 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:44.002128    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x7brk"
Sep 09 21:37:44.004430 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:44.004404    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-mb7j5\""
Sep 09 21:37:44.004430 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:44.004420    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 09 21:37:44.004586 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:44.004426    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 09 21:37:44.013732 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:44.013712    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x7brk"]
Sep 09 21:37:44.068075 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:44.068045    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfvgt\" (UniqueName: \"kubernetes.io/projected/989307ea-4c3f-4074-9d26-05ae25d39fb3-kube-api-access-jfvgt\") pod \"cert-manager-operator-controller-manager-55c444bf86-x7brk\" (UID: \"989307ea-4c3f-4074-9d26-05ae25d39fb3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x7brk"
Sep 09 21:37:44.068182 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:44.068092    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/989307ea-4c3f-4074-9d26-05ae25d39fb3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x7brk\" (UID: \"989307ea-4c3f-4074-9d26-05ae25d39fb3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x7brk"
Sep 09 21:37:44.168713 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:44.168682    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfvgt\" (UniqueName: \"kubernetes.io/projected/989307ea-4c3f-4074-9d26-05ae25d39fb3-kube-api-access-jfvgt\") pod \"cert-manager-operator-controller-manager-55c444bf86-x7brk\" (UID: \"989307ea-4c3f-4074-9d26-05ae25d39fb3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x7brk"
Sep 09 21:37:44.168862 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:44.168727    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/989307ea-4c3f-4074-9d26-05ae25d39fb3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x7brk\" (UID: \"989307ea-4c3f-4074-9d26-05ae25d39fb3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x7brk"
Sep 09 21:37:44.169137 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:44.169116    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/989307ea-4c3f-4074-9d26-05ae25d39fb3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x7brk\" (UID: \"989307ea-4c3f-4074-9d26-05ae25d39fb3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x7brk"
Sep 09 21:37:44.180197 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:44.180175    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfvgt\" (UniqueName: \"kubernetes.io/projected/989307ea-4c3f-4074-9d26-05ae25d39fb3-kube-api-access-jfvgt\") pod \"cert-manager-operator-controller-manager-55c444bf86-x7brk\" (UID: \"989307ea-4c3f-4074-9d26-05ae25d39fb3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x7brk"
Sep 09 21:37:44.311374 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:44.311353    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x7brk"
Sep 09 21:37:44.587766 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:44.587639    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x7brk"]
Sep 09 21:37:44.590586 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:37:44.590558    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod989307ea_4c3f_4074_9d26_05ae25d39fb3.slice/crio-47c90f8f320801aef3a312ba514a6e49087e9c66fa79fc2c7ddb22f7502aca46 WatchSource:0}: Error finding container 47c90f8f320801aef3a312ba514a6e49087e9c66fa79fc2c7ddb22f7502aca46: Status 404 returned error can't find the container with id 47c90f8f320801aef3a312ba514a6e49087e9c66fa79fc2c7ddb22f7502aca46
Sep 09 21:37:45.141165 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:45.141129    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x7brk" event={"ID":"989307ea-4c3f-4074-9d26-05ae25d39fb3","Type":"ContainerStarted","Data":"47c90f8f320801aef3a312ba514a6e49087e9c66fa79fc2c7ddb22f7502aca46"}
Sep 09 21:37:47.150372 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:47.150336    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x7brk" event={"ID":"989307ea-4c3f-4074-9d26-05ae25d39fb3","Type":"ContainerStarted","Data":"9bc746996bcf021618020803c95871ab85c2b37c0118e51350d758421677ad8d"}
Sep 09 21:37:47.169197 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:47.169141    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x7brk" podStartSLOduration=2.595645422 podStartE2EDuration="4.169125267s" totalImagesPullingTime="1.573479845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:37:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:37:47.168571499 +0000 UTC m=+286.641536408" watchObservedRunningTime="2026-09-09 21:37:47.169125267 +0000 UTC m=+286.642090176"
Sep 09 21:37:48.311302 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.311270    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"]
Sep 09 21:37:48.313602 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.313585    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"
Sep 09 21:37:48.315976 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.315946    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 21:37:48.316832 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.316813    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 21:37:48.316892 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.316815    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9cxbc\""
Sep 09 21:37:48.325285 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.325264    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"]
Sep 09 21:37:48.408110 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.408086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d1be494-eb64-4d12-8f53-f3653e66c563-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq\" (UID: \"5d1be494-eb64-4d12-8f53-f3653e66c563\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"
Sep 09 21:37:48.408220 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.408136    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c694\" (UniqueName: \"kubernetes.io/projected/5d1be494-eb64-4d12-8f53-f3653e66c563-kube-api-access-6c694\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq\" (UID: \"5d1be494-eb64-4d12-8f53-f3653e66c563\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"
Sep 09 21:37:48.408220 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.408183    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d1be494-eb64-4d12-8f53-f3653e66c563-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq\" (UID: \"5d1be494-eb64-4d12-8f53-f3653e66c563\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"
Sep 09 21:37:48.508713 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.508684    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c694\" (UniqueName: \"kubernetes.io/projected/5d1be494-eb64-4d12-8f53-f3653e66c563-kube-api-access-6c694\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq\" (UID: \"5d1be494-eb64-4d12-8f53-f3653e66c563\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"
Sep 09 21:37:48.508822 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.508729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d1be494-eb64-4d12-8f53-f3653e66c563-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq\" (UID: \"5d1be494-eb64-4d12-8f53-f3653e66c563\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"
Sep 09 21:37:48.508822 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.508753    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d1be494-eb64-4d12-8f53-f3653e66c563-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq\" (UID: \"5d1be494-eb64-4d12-8f53-f3653e66c563\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"
Sep 09 21:37:48.509061 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.509044    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d1be494-eb64-4d12-8f53-f3653e66c563-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq\" (UID: \"5d1be494-eb64-4d12-8f53-f3653e66c563\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"
Sep 09 21:37:48.509103 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.509078    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d1be494-eb64-4d12-8f53-f3653e66c563-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq\" (UID: \"5d1be494-eb64-4d12-8f53-f3653e66c563\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"
Sep 09 21:37:48.518762 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.518739    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c694\" (UniqueName: \"kubernetes.io/projected/5d1be494-eb64-4d12-8f53-f3653e66c563-kube-api-access-6c694\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq\" (UID: \"5d1be494-eb64-4d12-8f53-f3653e66c563\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"
Sep 09 21:37:48.622859 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.622809    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"
Sep 09 21:37:48.889618 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:48.889547    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"]
Sep 09 21:37:48.892364 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:37:48.892331    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d1be494_eb64_4d12_8f53_f3653e66c563.slice/crio-c094ac3ee6208081bb6290af28e983901927b2fa001459361be37c214816030d WatchSource:0}: Error finding container c094ac3ee6208081bb6290af28e983901927b2fa001459361be37c214816030d: Status 404 returned error can't find the container with id c094ac3ee6208081bb6290af28e983901927b2fa001459361be37c214816030d
Sep 09 21:37:49.160007 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:49.159940    2578 generic.go:356] "Generic (PLEG): container finished" podID="5d1be494-eb64-4d12-8f53-f3653e66c563" containerID="f692de6f6e08568e91788776e1d77904e69f73435cba4934cc4859ecca830fb3" exitCode=0
Sep 09 21:37:49.160007 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:49.159999    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq" event={"ID":"5d1be494-eb64-4d12-8f53-f3653e66c563","Type":"ContainerDied","Data":"f692de6f6e08568e91788776e1d77904e69f73435cba4934cc4859ecca830fb3"}
Sep 09 21:37:49.160161 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:49.160043    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq" event={"ID":"5d1be494-eb64-4d12-8f53-f3653e66c563","Type":"ContainerStarted","Data":"c094ac3ee6208081bb6290af28e983901927b2fa001459361be37c214816030d"}
Sep 09 21:37:52.175678 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:52.175643    2578 generic.go:356] "Generic (PLEG): container finished" podID="5d1be494-eb64-4d12-8f53-f3653e66c563" containerID="02a693b83d16187a9a2e6d5a2e632f9aa525a4bd9a6fdb847addacd4cc5ca501" exitCode=0
Sep 09 21:37:52.176085 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:52.175729    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq" event={"ID":"5d1be494-eb64-4d12-8f53-f3653e66c563","Type":"ContainerDied","Data":"02a693b83d16187a9a2e6d5a2e632f9aa525a4bd9a6fdb847addacd4cc5ca501"}
Sep 09 21:37:53.181278 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:53.181240    2578 generic.go:356] "Generic (PLEG): container finished" podID="5d1be494-eb64-4d12-8f53-f3653e66c563" containerID="1ae748601ebd1583fa98e02b52e84d1d288a9439c6741e3f003ac1e64c6ee2fc" exitCode=0
Sep 09 21:37:53.181669 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:53.181321    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq" event={"ID":"5d1be494-eb64-4d12-8f53-f3653e66c563","Type":"ContainerDied","Data":"1ae748601ebd1583fa98e02b52e84d1d288a9439c6741e3f003ac1e64c6ee2fc"}
Sep 09 21:37:54.312531 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.312509    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"
Sep 09 21:37:54.454827 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.454768    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d1be494-eb64-4d12-8f53-f3653e66c563-bundle\") pod \"5d1be494-eb64-4d12-8f53-f3653e66c563\" (UID: \"5d1be494-eb64-4d12-8f53-f3653e66c563\") "
Sep 09 21:37:54.454827 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.454801    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6c694\" (UniqueName: \"kubernetes.io/projected/5d1be494-eb64-4d12-8f53-f3653e66c563-kube-api-access-6c694\") pod \"5d1be494-eb64-4d12-8f53-f3653e66c563\" (UID: \"5d1be494-eb64-4d12-8f53-f3653e66c563\") "
Sep 09 21:37:54.455000 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.454833    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d1be494-eb64-4d12-8f53-f3653e66c563-util\") pod \"5d1be494-eb64-4d12-8f53-f3653e66c563\" (UID: \"5d1be494-eb64-4d12-8f53-f3653e66c563\") "
Sep 09 21:37:54.455239 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.455210    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d1be494-eb64-4d12-8f53-f3653e66c563-bundle" (OuterVolumeSpecName: "bundle") pod "5d1be494-eb64-4d12-8f53-f3653e66c563" (UID: "5d1be494-eb64-4d12-8f53-f3653e66c563"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:37:54.456850 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.456828    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d1be494-eb64-4d12-8f53-f3653e66c563-kube-api-access-6c694" (OuterVolumeSpecName: "kube-api-access-6c694") pod "5d1be494-eb64-4d12-8f53-f3653e66c563" (UID: "5d1be494-eb64-4d12-8f53-f3653e66c563"). InnerVolumeSpecName "kube-api-access-6c694". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:37:54.460055 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.460032    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d1be494-eb64-4d12-8f53-f3653e66c563-util" (OuterVolumeSpecName: "util") pod "5d1be494-eb64-4d12-8f53-f3653e66c563" (UID: "5d1be494-eb64-4d12-8f53-f3653e66c563"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:37:54.555268 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.555245    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d1be494-eb64-4d12-8f53-f3653e66c563-bundle\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:37:54.555268 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.555265    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6c694\" (UniqueName: \"kubernetes.io/projected/5d1be494-eb64-4d12-8f53-f3653e66c563-kube-api-access-6c694\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:37:54.555382 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.555276    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d1be494-eb64-4d12-8f53-f3653e66c563-util\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:37:54.953261 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.953235    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f8bf8"]
Sep 09 21:37:54.953510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.953497    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d1be494-eb64-4d12-8f53-f3653e66c563" containerName="pull"
Sep 09 21:37:54.953600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.953512    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d1be494-eb64-4d12-8f53-f3653e66c563" containerName="pull"
Sep 09 21:37:54.953600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.953524    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d1be494-eb64-4d12-8f53-f3653e66c563" containerName="extract"
Sep 09 21:37:54.953600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.953529    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d1be494-eb64-4d12-8f53-f3653e66c563" containerName="extract"
Sep 09 21:37:54.953600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.953545    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d1be494-eb64-4d12-8f53-f3653e66c563" containerName="util"
Sep 09 21:37:54.953600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.953550    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d1be494-eb64-4d12-8f53-f3653e66c563" containerName="util"
Sep 09 21:37:54.953600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.953596    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d1be494-eb64-4d12-8f53-f3653e66c563" containerName="extract"
Sep 09 21:37:54.957115 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.957102    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8bf8"
Sep 09 21:37:54.959336 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.959319    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 09 21:37:54.959432 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.959340    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-7mvpv\""
Sep 09 21:37:54.960367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.960352    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 09 21:37:54.963023 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:54.963001    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f8bf8"]
Sep 09 21:37:55.058086 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:55.058061    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2953f45a-ce40-43ff-8135-b8e4b7f15384-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f8bf8\" (UID: \"2953f45a-ce40-43ff-8135-b8e4b7f15384\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8bf8"
Sep 09 21:37:55.058165 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:55.058096    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmff5\" (UniqueName: \"kubernetes.io/projected/2953f45a-ce40-43ff-8135-b8e4b7f15384-kube-api-access-dmff5\") pod \"cert-manager-cainjector-559cdd8c87-f8bf8\" (UID: \"2953f45a-ce40-43ff-8135-b8e4b7f15384\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8bf8"
Sep 09 21:37:55.158450 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:55.158425    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2953f45a-ce40-43ff-8135-b8e4b7f15384-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f8bf8\" (UID: \"2953f45a-ce40-43ff-8135-b8e4b7f15384\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8bf8"
Sep 09 21:37:55.158540 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:55.158461    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmff5\" (UniqueName: \"kubernetes.io/projected/2953f45a-ce40-43ff-8135-b8e4b7f15384-kube-api-access-dmff5\") pod \"cert-manager-cainjector-559cdd8c87-f8bf8\" (UID: \"2953f45a-ce40-43ff-8135-b8e4b7f15384\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8bf8"
Sep 09 21:37:55.166681 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:55.166653    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2953f45a-ce40-43ff-8135-b8e4b7f15384-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f8bf8\" (UID: \"2953f45a-ce40-43ff-8135-b8e4b7f15384\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8bf8"
Sep 09 21:37:55.166854 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:55.166834    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmff5\" (UniqueName: \"kubernetes.io/projected/2953f45a-ce40-43ff-8135-b8e4b7f15384-kube-api-access-dmff5\") pod \"cert-manager-cainjector-559cdd8c87-f8bf8\" (UID: \"2953f45a-ce40-43ff-8135-b8e4b7f15384\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8bf8"
Sep 09 21:37:55.193437 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:55.193413    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq" event={"ID":"5d1be494-eb64-4d12-8f53-f3653e66c563","Type":"ContainerDied","Data":"c094ac3ee6208081bb6290af28e983901927b2fa001459361be37c214816030d"}
Sep 09 21:37:55.193554 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:55.193440    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzgcjq"
Sep 09 21:37:55.193554 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:55.193457    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c094ac3ee6208081bb6290af28e983901927b2fa001459361be37c214816030d"
Sep 09 21:37:55.274849 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:55.274827    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8bf8"
Sep 09 21:37:55.543161 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:55.543120    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f8bf8"]
Sep 09 21:37:55.545599 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:37:55.545561    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2953f45a_ce40_43ff_8135_b8e4b7f15384.slice/crio-20c23e79350079700deca292531ddb86305607e662b1211d7562be306e1a3d8a WatchSource:0}: Error finding container 20c23e79350079700deca292531ddb86305607e662b1211d7562be306e1a3d8a: Status 404 returned error can't find the container with id 20c23e79350079700deca292531ddb86305607e662b1211d7562be306e1a3d8a
Sep 09 21:37:56.198902 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:56.198866    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8bf8" event={"ID":"2953f45a-ce40-43ff-8135-b8e4b7f15384","Type":"ContainerStarted","Data":"20c23e79350079700deca292531ddb86305607e662b1211d7562be306e1a3d8a"}
Sep 09 21:37:58.221728 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:58.221692    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c5867"]
Sep 09 21:37:58.224948 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:58.224925    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c5867"
Sep 09 21:37:58.227221 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:58.227190    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-mc8z4\""
Sep 09 21:37:58.232488 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:58.232467    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c5867"]
Sep 09 21:37:58.389683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:58.389648    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3497560-77c5-47f3-875b-811f57cfd3d9-bound-sa-token\") pod \"cert-manager-55b4d954d5-c5867\" (UID: \"f3497560-77c5-47f3-875b-811f57cfd3d9\") " pod="cert-manager/cert-manager-55b4d954d5-c5867"
Sep 09 21:37:58.389822 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:58.389716    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m6dx\" (UniqueName: \"kubernetes.io/projected/f3497560-77c5-47f3-875b-811f57cfd3d9-kube-api-access-2m6dx\") pod \"cert-manager-55b4d954d5-c5867\" (UID: \"f3497560-77c5-47f3-875b-811f57cfd3d9\") " pod="cert-manager/cert-manager-55b4d954d5-c5867"
Sep 09 21:37:58.490637 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:58.490609    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3497560-77c5-47f3-875b-811f57cfd3d9-bound-sa-token\") pod \"cert-manager-55b4d954d5-c5867\" (UID: \"f3497560-77c5-47f3-875b-811f57cfd3d9\") " pod="cert-manager/cert-manager-55b4d954d5-c5867"
Sep 09 21:37:58.490731 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:58.490664    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m6dx\" (UniqueName: \"kubernetes.io/projected/f3497560-77c5-47f3-875b-811f57cfd3d9-kube-api-access-2m6dx\") pod \"cert-manager-55b4d954d5-c5867\" (UID: \"f3497560-77c5-47f3-875b-811f57cfd3d9\") " pod="cert-manager/cert-manager-55b4d954d5-c5867"
Sep 09 21:37:58.498942 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:58.498911    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m6dx\" (UniqueName: \"kubernetes.io/projected/f3497560-77c5-47f3-875b-811f57cfd3d9-kube-api-access-2m6dx\") pod \"cert-manager-55b4d954d5-c5867\" (UID: \"f3497560-77c5-47f3-875b-811f57cfd3d9\") " pod="cert-manager/cert-manager-55b4d954d5-c5867"
Sep 09 21:37:58.498942 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:58.498928    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3497560-77c5-47f3-875b-811f57cfd3d9-bound-sa-token\") pod \"cert-manager-55b4d954d5-c5867\" (UID: \"f3497560-77c5-47f3-875b-811f57cfd3d9\") " pod="cert-manager/cert-manager-55b4d954d5-c5867"
Sep 09 21:37:58.536848 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:58.536822    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c5867"
Sep 09 21:37:58.808095 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:58.808058    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c5867"]
Sep 09 21:37:58.810712 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:37:58.810690    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3497560_77c5_47f3_875b_811f57cfd3d9.slice/crio-7b988b789856f64a441d640e7492bfde2608b0cec2d7d741c2f89bd0bd0ed20f WatchSource:0}: Error finding container 7b988b789856f64a441d640e7492bfde2608b0cec2d7d741c2f89bd0bd0ed20f: Status 404 returned error can't find the container with id 7b988b789856f64a441d640e7492bfde2608b0cec2d7d741c2f89bd0bd0ed20f
Sep 09 21:37:59.215150 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:59.215062    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8bf8" event={"ID":"2953f45a-ce40-43ff-8135-b8e4b7f15384","Type":"ContainerStarted","Data":"a4965dd020d89785079515da05cb70e5bac17092c05ebc06a8db34624b1b2bef"}
Sep 09 21:37:59.216600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:59.216575    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c5867" event={"ID":"f3497560-77c5-47f3-875b-811f57cfd3d9","Type":"ContainerStarted","Data":"ced8c0dd5ab59ffaa103f2595a0efe91998a4085ec0684dcc8f2fed3f5a3d8ce"}
Sep 09 21:37:59.216704 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:59.216609    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c5867" event={"ID":"f3497560-77c5-47f3-875b-811f57cfd3d9","Type":"ContainerStarted","Data":"7b988b789856f64a441d640e7492bfde2608b0cec2d7d741c2f89bd0bd0ed20f"}
Sep 09 21:37:59.231832 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:59.231774    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f8bf8" podStartSLOduration=2.35187176 podStartE2EDuration="5.231761252s" totalImagesPullingTime="2.879889492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:37:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:37:59.229604361 +0000 UTC m=+298.702569272" watchObservedRunningTime="2026-09-09 21:37:59.231761252 +0000 UTC m=+298.704726162"
Sep 09 21:37:59.246131 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:37:59.246090    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-c5867" podStartSLOduration=1.246077168 podStartE2EDuration="1.246077168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:37:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:37:59.244309851 +0000 UTC m=+298.717274763" watchObservedRunningTime="2026-09-09 21:37:59.246077168 +0000 UTC m=+298.719042078"
Sep 09 21:38:00.915950 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:00.915924    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 21:38:00.916468 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:00.916040    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 21:38:00.927447 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:00.927418    2578 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 21:38:05.227548 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.227518    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"]
Sep 09 21:38:05.272424 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.272396    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"]
Sep 09 21:38:05.272520 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.272493    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"
Sep 09 21:38:05.276703 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.276679    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 21:38:05.277402 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.277385    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 21:38:05.277515 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.277385    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9cxbc\""
Sep 09 21:38:05.340425 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.340398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-566hp\" (UniqueName: \"kubernetes.io/projected/b7d4f00c-2177-4a30-9880-56e1fae9116e-kube-api-access-566hp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc\" (UID: \"b7d4f00c-2177-4a30-9880-56e1fae9116e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"
Sep 09 21:38:05.340534 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.340427    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7d4f00c-2177-4a30-9880-56e1fae9116e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc\" (UID: \"b7d4f00c-2177-4a30-9880-56e1fae9116e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"
Sep 09 21:38:05.340534 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.340474    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7d4f00c-2177-4a30-9880-56e1fae9116e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc\" (UID: \"b7d4f00c-2177-4a30-9880-56e1fae9116e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"
Sep 09 21:38:05.441707 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.441683    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-566hp\" (UniqueName: \"kubernetes.io/projected/b7d4f00c-2177-4a30-9880-56e1fae9116e-kube-api-access-566hp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc\" (UID: \"b7d4f00c-2177-4a30-9880-56e1fae9116e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"
Sep 09 21:38:05.441801 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.441712    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7d4f00c-2177-4a30-9880-56e1fae9116e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc\" (UID: \"b7d4f00c-2177-4a30-9880-56e1fae9116e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"
Sep 09 21:38:05.441801 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.441766    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7d4f00c-2177-4a30-9880-56e1fae9116e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc\" (UID: \"b7d4f00c-2177-4a30-9880-56e1fae9116e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"
Sep 09 21:38:05.442140 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.442123    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7d4f00c-2177-4a30-9880-56e1fae9116e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc\" (UID: \"b7d4f00c-2177-4a30-9880-56e1fae9116e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"
Sep 09 21:38:05.442185 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.442142    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7d4f00c-2177-4a30-9880-56e1fae9116e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc\" (UID: \"b7d4f00c-2177-4a30-9880-56e1fae9116e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"
Sep 09 21:38:05.449181 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.449163    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-566hp\" (UniqueName: \"kubernetes.io/projected/b7d4f00c-2177-4a30-9880-56e1fae9116e-kube-api-access-566hp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc\" (UID: \"b7d4f00c-2177-4a30-9880-56e1fae9116e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"
Sep 09 21:38:05.581797 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.581776    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"
Sep 09 21:38:05.864067 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:05.864037    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"]
Sep 09 21:38:05.866648 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:38:05.866620    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d4f00c_2177_4a30_9880_56e1fae9116e.slice/crio-9901577c44fa77068613d7b2fe35df6b76a912e323f43dd6f74027351f182c81 WatchSource:0}: Error finding container 9901577c44fa77068613d7b2fe35df6b76a912e323f43dd6f74027351f182c81: Status 404 returned error can't find the container with id 9901577c44fa77068613d7b2fe35df6b76a912e323f43dd6f74027351f182c81
Sep 09 21:38:06.246101 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:06.246007    2578 generic.go:356] "Generic (PLEG): container finished" podID="b7d4f00c-2177-4a30-9880-56e1fae9116e" containerID="9a077aec1145145ac0ce2b355d2dc154f4e6fee24f9535fc1683e8d16b4f5ba0" exitCode=0
Sep 09 21:38:06.246515 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:06.246093    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc" event={"ID":"b7d4f00c-2177-4a30-9880-56e1fae9116e","Type":"ContainerDied","Data":"9a077aec1145145ac0ce2b355d2dc154f4e6fee24f9535fc1683e8d16b4f5ba0"}
Sep 09 21:38:06.246515 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:06.246175    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc" event={"ID":"b7d4f00c-2177-4a30-9880-56e1fae9116e","Type":"ContainerStarted","Data":"9901577c44fa77068613d7b2fe35df6b76a912e323f43dd6f74027351f182c81"}
Sep 09 21:38:06.246515 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:06.246510    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 21:38:07.252381 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:07.252309    2578 generic.go:356] "Generic (PLEG): container finished" podID="b7d4f00c-2177-4a30-9880-56e1fae9116e" containerID="6eee1128d3094aae8c09f768e70bb7dab02e440dc53e0724e0b7cceccb8f7a0b" exitCode=0
Sep 09 21:38:07.252381 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:07.252355    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc" event={"ID":"b7d4f00c-2177-4a30-9880-56e1fae9116e","Type":"ContainerDied","Data":"6eee1128d3094aae8c09f768e70bb7dab02e440dc53e0724e0b7cceccb8f7a0b"}
Sep 09 21:38:08.258759 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:08.258724    2578 generic.go:356] "Generic (PLEG): container finished" podID="b7d4f00c-2177-4a30-9880-56e1fae9116e" containerID="1ff098ff81424933c987ded092315c592541eb83658fb525185ca3886eee9b90" exitCode=0
Sep 09 21:38:08.259182 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:08.258814    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc" event={"ID":"b7d4f00c-2177-4a30-9880-56e1fae9116e","Type":"ContainerDied","Data":"1ff098ff81424933c987ded092315c592541eb83658fb525185ca3886eee9b90"}
Sep 09 21:38:09.386783 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:09.386762    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"
Sep 09 21:38:09.472664 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:09.472637    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7d4f00c-2177-4a30-9880-56e1fae9116e-util\") pod \"b7d4f00c-2177-4a30-9880-56e1fae9116e\" (UID: \"b7d4f00c-2177-4a30-9880-56e1fae9116e\") "
Sep 09 21:38:09.472791 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:09.472686    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-566hp\" (UniqueName: \"kubernetes.io/projected/b7d4f00c-2177-4a30-9880-56e1fae9116e-kube-api-access-566hp\") pod \"b7d4f00c-2177-4a30-9880-56e1fae9116e\" (UID: \"b7d4f00c-2177-4a30-9880-56e1fae9116e\") "
Sep 09 21:38:09.472791 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:09.472705    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7d4f00c-2177-4a30-9880-56e1fae9116e-bundle\") pod \"b7d4f00c-2177-4a30-9880-56e1fae9116e\" (UID: \"b7d4f00c-2177-4a30-9880-56e1fae9116e\") "
Sep 09 21:38:09.473433 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:09.473401    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d4f00c-2177-4a30-9880-56e1fae9116e-bundle" (OuterVolumeSpecName: "bundle") pod "b7d4f00c-2177-4a30-9880-56e1fae9116e" (UID: "b7d4f00c-2177-4a30-9880-56e1fae9116e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:38:09.474731 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:09.474703    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7d4f00c-2177-4a30-9880-56e1fae9116e-kube-api-access-566hp" (OuterVolumeSpecName: "kube-api-access-566hp") pod "b7d4f00c-2177-4a30-9880-56e1fae9116e" (UID: "b7d4f00c-2177-4a30-9880-56e1fae9116e"). InnerVolumeSpecName "kube-api-access-566hp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:38:09.478135 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:09.478115    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d4f00c-2177-4a30-9880-56e1fae9116e-util" (OuterVolumeSpecName: "util") pod "b7d4f00c-2177-4a30-9880-56e1fae9116e" (UID: "b7d4f00c-2177-4a30-9880-56e1fae9116e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:38:09.573733 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:09.573711    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-566hp\" (UniqueName: \"kubernetes.io/projected/b7d4f00c-2177-4a30-9880-56e1fae9116e-kube-api-access-566hp\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:38:09.573733 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:09.573732    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7d4f00c-2177-4a30-9880-56e1fae9116e-bundle\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:38:09.573868 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:09.573742    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7d4f00c-2177-4a30-9880-56e1fae9116e-util\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:38:10.268407 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:10.268380    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc"
Sep 09 21:38:10.268572 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:10.268379    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmv5gc" event={"ID":"b7d4f00c-2177-4a30-9880-56e1fae9116e","Type":"ContainerDied","Data":"9901577c44fa77068613d7b2fe35df6b76a912e323f43dd6f74027351f182c81"}
Sep 09 21:38:10.268572 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:10.268500    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9901577c44fa77068613d7b2fe35df6b76a912e323f43dd6f74027351f182c81"
Sep 09 21:38:19.296235 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.296199    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"]
Sep 09 21:38:19.296685 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.296480    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7d4f00c-2177-4a30-9880-56e1fae9116e" containerName="pull"
Sep 09 21:38:19.296685 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.296490    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7d4f00c-2177-4a30-9880-56e1fae9116e" containerName="pull"
Sep 09 21:38:19.296685 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.296500    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7d4f00c-2177-4a30-9880-56e1fae9116e" containerName="extract"
Sep 09 21:38:19.296685 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.296506    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7d4f00c-2177-4a30-9880-56e1fae9116e" containerName="extract"
Sep 09 21:38:19.296685 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.296517    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7d4f00c-2177-4a30-9880-56e1fae9116e" containerName="util"
Sep 09 21:38:19.296685 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.296523    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7d4f00c-2177-4a30-9880-56e1fae9116e" containerName="util"
Sep 09 21:38:19.296685 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.296570    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7d4f00c-2177-4a30-9880-56e1fae9116e" containerName="extract"
Sep 09 21:38:19.299612 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.299596    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:19.302698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.302672    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 09 21:38:19.303823 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.303678    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 09 21:38:19.303823 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.303689    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 09 21:38:19.303823 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.303685    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 09 21:38:19.303823 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.303685    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 09 21:38:19.303823 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.303754    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-gxjm8\""
Sep 09 21:38:19.307783 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.307760    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"]
Sep 09 21:38:19.342388 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.342360    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3ff621b4-7fe0-496a-a072-b41fbd411abc-manager-config\") pod \"lws-controller-manager-64d94cf4d-hjk2v\" (UID: \"3ff621b4-7fe0-496a-a072-b41fbd411abc\") " pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:19.342513 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.342414    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ff621b4-7fe0-496a-a072-b41fbd411abc-cert\") pod \"lws-controller-manager-64d94cf4d-hjk2v\" (UID: \"3ff621b4-7fe0-496a-a072-b41fbd411abc\") " pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:19.342513 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.342472    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3ff621b4-7fe0-496a-a072-b41fbd411abc-metrics-cert\") pod \"lws-controller-manager-64d94cf4d-hjk2v\" (UID: \"3ff621b4-7fe0-496a-a072-b41fbd411abc\") " pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:19.342513 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.342507    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xzhk\" (UniqueName: \"kubernetes.io/projected/3ff621b4-7fe0-496a-a072-b41fbd411abc-kube-api-access-5xzhk\") pod \"lws-controller-manager-64d94cf4d-hjk2v\" (UID: \"3ff621b4-7fe0-496a-a072-b41fbd411abc\") " pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:19.443563 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.443540    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3ff621b4-7fe0-496a-a072-b41fbd411abc-manager-config\") pod \"lws-controller-manager-64d94cf4d-hjk2v\" (UID: \"3ff621b4-7fe0-496a-a072-b41fbd411abc\") " pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:19.443682 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.443582    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ff621b4-7fe0-496a-a072-b41fbd411abc-cert\") pod \"lws-controller-manager-64d94cf4d-hjk2v\" (UID: \"3ff621b4-7fe0-496a-a072-b41fbd411abc\") " pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:19.443682 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.443604    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3ff621b4-7fe0-496a-a072-b41fbd411abc-metrics-cert\") pod \"lws-controller-manager-64d94cf4d-hjk2v\" (UID: \"3ff621b4-7fe0-496a-a072-b41fbd411abc\") " pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:19.443682 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.443631    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xzhk\" (UniqueName: \"kubernetes.io/projected/3ff621b4-7fe0-496a-a072-b41fbd411abc-kube-api-access-5xzhk\") pod \"lws-controller-manager-64d94cf4d-hjk2v\" (UID: \"3ff621b4-7fe0-496a-a072-b41fbd411abc\") " pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:19.444234 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.444214    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3ff621b4-7fe0-496a-a072-b41fbd411abc-manager-config\") pod \"lws-controller-manager-64d94cf4d-hjk2v\" (UID: \"3ff621b4-7fe0-496a-a072-b41fbd411abc\") " pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:19.446134 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.446113    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3ff621b4-7fe0-496a-a072-b41fbd411abc-metrics-cert\") pod \"lws-controller-manager-64d94cf4d-hjk2v\" (UID: \"3ff621b4-7fe0-496a-a072-b41fbd411abc\") " pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:19.446220 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.446172    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ff621b4-7fe0-496a-a072-b41fbd411abc-cert\") pod \"lws-controller-manager-64d94cf4d-hjk2v\" (UID: \"3ff621b4-7fe0-496a-a072-b41fbd411abc\") " pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:19.451336 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.451313    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xzhk\" (UniqueName: \"kubernetes.io/projected/3ff621b4-7fe0-496a-a072-b41fbd411abc-kube-api-access-5xzhk\") pod \"lws-controller-manager-64d94cf4d-hjk2v\" (UID: \"3ff621b4-7fe0-496a-a072-b41fbd411abc\") " pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:19.609942 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.609887    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:19.887005 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:19.886908    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"]
Sep 09 21:38:19.892096 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:38:19.892056    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff621b4_7fe0_496a_a072_b41fbd411abc.slice/crio-723cbdb89f4d1f08722bd4d2a008e1b45a54d52fb73cf14a6481dbc5bc6e7146 WatchSource:0}: Error finding container 723cbdb89f4d1f08722bd4d2a008e1b45a54d52fb73cf14a6481dbc5bc6e7146: Status 404 returned error can't find the container with id 723cbdb89f4d1f08722bd4d2a008e1b45a54d52fb73cf14a6481dbc5bc6e7146
Sep 09 21:38:20.311464 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:20.311429    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v" event={"ID":"3ff621b4-7fe0-496a-a072-b41fbd411abc","Type":"ContainerStarted","Data":"723cbdb89f4d1f08722bd4d2a008e1b45a54d52fb73cf14a6481dbc5bc6e7146"}
Sep 09 21:38:21.460876 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.460848    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"]
Sep 09 21:38:21.463350 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.463330    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"
Sep 09 21:38:21.465631 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.465608    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 21:38:21.465776 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.465694    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 21:38:21.466848 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.466827    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9cxbc\""
Sep 09 21:38:21.472902 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.472881    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"]
Sep 09 21:38:21.561096 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.561072    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6dec92b-3600-4d2b-a578-92b4382aca7d-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7\" (UID: \"e6dec92b-3600-4d2b-a578-92b4382aca7d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"
Sep 09 21:38:21.561204 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.561137    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6dec92b-3600-4d2b-a578-92b4382aca7d-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7\" (UID: \"e6dec92b-3600-4d2b-a578-92b4382aca7d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"
Sep 09 21:38:21.561204 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.561197    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg4dg\" (UniqueName: \"kubernetes.io/projected/e6dec92b-3600-4d2b-a578-92b4382aca7d-kube-api-access-sg4dg\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7\" (UID: \"e6dec92b-3600-4d2b-a578-92b4382aca7d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"
Sep 09 21:38:21.662233 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.662202    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6dec92b-3600-4d2b-a578-92b4382aca7d-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7\" (UID: \"e6dec92b-3600-4d2b-a578-92b4382aca7d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"
Sep 09 21:38:21.662359 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.662244    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg4dg\" (UniqueName: \"kubernetes.io/projected/e6dec92b-3600-4d2b-a578-92b4382aca7d-kube-api-access-sg4dg\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7\" (UID: \"e6dec92b-3600-4d2b-a578-92b4382aca7d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"
Sep 09 21:38:21.662359 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.662265    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6dec92b-3600-4d2b-a578-92b4382aca7d-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7\" (UID: \"e6dec92b-3600-4d2b-a578-92b4382aca7d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"
Sep 09 21:38:21.662639 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.662618    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6dec92b-3600-4d2b-a578-92b4382aca7d-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7\" (UID: \"e6dec92b-3600-4d2b-a578-92b4382aca7d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"
Sep 09 21:38:21.662639 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.662630    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6dec92b-3600-4d2b-a578-92b4382aca7d-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7\" (UID: \"e6dec92b-3600-4d2b-a578-92b4382aca7d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"
Sep 09 21:38:21.675826 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.675797    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg4dg\" (UniqueName: \"kubernetes.io/projected/e6dec92b-3600-4d2b-a578-92b4382aca7d-kube-api-access-sg4dg\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7\" (UID: \"e6dec92b-3600-4d2b-a578-92b4382aca7d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"
Sep 09 21:38:21.704920 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.704893    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"]
Sep 09 21:38:21.707720 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.707700    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:21.710110 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.710092    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 09 21:38:21.710210 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.710191    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 09 21:38:21.710493 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.710469    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9n6ch\""
Sep 09 21:38:21.710575 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.710513    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 09 21:38:21.710575 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.710533    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 09 21:38:21.727310 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.727290    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"]
Sep 09 21:38:21.763127 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.763100    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03067d81-ba5a-44ca-abe3-6c5f9f1e166c-tmp\") pod \"opendatahub-operator-controller-manager-84fbbd4d59-wh2vj\" (UID: \"03067d81-ba5a-44ca-abe3-6c5f9f1e166c\") " pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:21.763256 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.763138    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/03067d81-ba5a-44ca-abe3-6c5f9f1e166c-webhook-cert\") pod \"opendatahub-operator-controller-manager-84fbbd4d59-wh2vj\" (UID: \"03067d81-ba5a-44ca-abe3-6c5f9f1e166c\") " pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:21.763256 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.763158    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cb4l\" (UniqueName: \"kubernetes.io/projected/03067d81-ba5a-44ca-abe3-6c5f9f1e166c-kube-api-access-2cb4l\") pod \"opendatahub-operator-controller-manager-84fbbd4d59-wh2vj\" (UID: \"03067d81-ba5a-44ca-abe3-6c5f9f1e166c\") " pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:21.763342 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.763252    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/03067d81-ba5a-44ca-abe3-6c5f9f1e166c-manifests\") pod \"opendatahub-operator-controller-manager-84fbbd4d59-wh2vj\" (UID: \"03067d81-ba5a-44ca-abe3-6c5f9f1e166c\") " pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:21.763342 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.763296    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/03067d81-ba5a-44ca-abe3-6c5f9f1e166c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84fbbd4d59-wh2vj\" (UID: \"03067d81-ba5a-44ca-abe3-6c5f9f1e166c\") " pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:21.773055 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.773033    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"
Sep 09 21:38:21.864500 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.864473    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/03067d81-ba5a-44ca-abe3-6c5f9f1e166c-manifests\") pod \"opendatahub-operator-controller-manager-84fbbd4d59-wh2vj\" (UID: \"03067d81-ba5a-44ca-abe3-6c5f9f1e166c\") " pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:21.864605 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.864517    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/03067d81-ba5a-44ca-abe3-6c5f9f1e166c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84fbbd4d59-wh2vj\" (UID: \"03067d81-ba5a-44ca-abe3-6c5f9f1e166c\") " pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:21.864605 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.864555    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03067d81-ba5a-44ca-abe3-6c5f9f1e166c-tmp\") pod \"opendatahub-operator-controller-manager-84fbbd4d59-wh2vj\" (UID: \"03067d81-ba5a-44ca-abe3-6c5f9f1e166c\") " pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:21.864605 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.864587    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/03067d81-ba5a-44ca-abe3-6c5f9f1e166c-webhook-cert\") pod \"opendatahub-operator-controller-manager-84fbbd4d59-wh2vj\" (UID: \"03067d81-ba5a-44ca-abe3-6c5f9f1e166c\") " pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:21.864779 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.864756    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cb4l\" (UniqueName: \"kubernetes.io/projected/03067d81-ba5a-44ca-abe3-6c5f9f1e166c-kube-api-access-2cb4l\") pod \"opendatahub-operator-controller-manager-84fbbd4d59-wh2vj\" (UID: \"03067d81-ba5a-44ca-abe3-6c5f9f1e166c\") " pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:21.864903 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.864880    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/03067d81-ba5a-44ca-abe3-6c5f9f1e166c-manifests\") pod \"opendatahub-operator-controller-manager-84fbbd4d59-wh2vj\" (UID: \"03067d81-ba5a-44ca-abe3-6c5f9f1e166c\") " pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:21.866779 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.866748    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03067d81-ba5a-44ca-abe3-6c5f9f1e166c-tmp\") pod \"opendatahub-operator-controller-manager-84fbbd4d59-wh2vj\" (UID: \"03067d81-ba5a-44ca-abe3-6c5f9f1e166c\") " pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:21.866878 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.866858    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/03067d81-ba5a-44ca-abe3-6c5f9f1e166c-webhook-cert\") pod \"opendatahub-operator-controller-manager-84fbbd4d59-wh2vj\" (UID: \"03067d81-ba5a-44ca-abe3-6c5f9f1e166c\") " pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:21.866931 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.866882    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/03067d81-ba5a-44ca-abe3-6c5f9f1e166c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84fbbd4d59-wh2vj\" (UID: \"03067d81-ba5a-44ca-abe3-6c5f9f1e166c\") " pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:21.872066 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:21.872048    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cb4l\" (UniqueName: \"kubernetes.io/projected/03067d81-ba5a-44ca-abe3-6c5f9f1e166c-kube-api-access-2cb4l\") pod \"opendatahub-operator-controller-manager-84fbbd4d59-wh2vj\" (UID: \"03067d81-ba5a-44ca-abe3-6c5f9f1e166c\") " pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:22.017938 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:22.017914    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:22.043487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:22.043467    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"]
Sep 09 21:38:22.045891 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:38:22.045862    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6dec92b_3600_4d2b_a578_92b4382aca7d.slice/crio-e3e007a449f49a6a8c10da4149380406a8305cecc195c2906bcb5d664a92b40e WatchSource:0}: Error finding container e3e007a449f49a6a8c10da4149380406a8305cecc195c2906bcb5d664a92b40e: Status 404 returned error can't find the container with id e3e007a449f49a6a8c10da4149380406a8305cecc195c2906bcb5d664a92b40e
Sep 09 21:38:22.301993 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:22.301903    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"]
Sep 09 21:38:22.305089 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:38:22.305062    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03067d81_ba5a_44ca_abe3_6c5f9f1e166c.slice/crio-9ba45439f76da74550687916a368b5af2cfac0fc34e3248d1aa8699ebaf910e6 WatchSource:0}: Error finding container 9ba45439f76da74550687916a368b5af2cfac0fc34e3248d1aa8699ebaf910e6: Status 404 returned error can't find the container with id 9ba45439f76da74550687916a368b5af2cfac0fc34e3248d1aa8699ebaf910e6
Sep 09 21:38:22.321484 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:22.321453    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v" event={"ID":"3ff621b4-7fe0-496a-a072-b41fbd411abc","Type":"ContainerStarted","Data":"2d087dabda429648ea987ec3c7e5dfdf3836c4362cde7ac61f4a0431a59f6fec"}
Sep 09 21:38:22.321592 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:22.321534    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:22.322990 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:22.322954    2578 generic.go:356] "Generic (PLEG): container finished" podID="e6dec92b-3600-4d2b-a578-92b4382aca7d" containerID="ced0fb3b8e7228eff4eee2c178d248fcffe8fcd54d71399bd214fc12312f9317" exitCode=0
Sep 09 21:38:22.323082 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:22.323021    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7" event={"ID":"e6dec92b-3600-4d2b-a578-92b4382aca7d","Type":"ContainerDied","Data":"ced0fb3b8e7228eff4eee2c178d248fcffe8fcd54d71399bd214fc12312f9317"}
Sep 09 21:38:22.323082 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:22.323061    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7" event={"ID":"e6dec92b-3600-4d2b-a578-92b4382aca7d","Type":"ContainerStarted","Data":"e3e007a449f49a6a8c10da4149380406a8305cecc195c2906bcb5d664a92b40e"}
Sep 09 21:38:22.324405 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:22.324383    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj" event={"ID":"03067d81-ba5a-44ca-abe3-6c5f9f1e166c","Type":"ContainerStarted","Data":"9ba45439f76da74550687916a368b5af2cfac0fc34e3248d1aa8699ebaf910e6"}
Sep 09 21:38:22.340092 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:22.340052    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v" podStartSLOduration=1.738822693 podStartE2EDuration="3.340038974s" totalImagesPullingTime="1.601216281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:38:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:38:22.337895402 +0000 UTC m=+321.810860315" watchObservedRunningTime="2026-09-09 21:38:22.340038974 +0000 UTC m=+321.813003884"
Sep 09 21:38:23.333614 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:23.333130    2578 generic.go:356] "Generic (PLEG): container finished" podID="e6dec92b-3600-4d2b-a578-92b4382aca7d" containerID="d878e5f2a1e518a5ab973daca1b339066f59dbd784d21bfb639d8f9eef5d5b89" exitCode=0
Sep 09 21:38:23.333614 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:23.333316    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7" event={"ID":"e6dec92b-3600-4d2b-a578-92b4382aca7d","Type":"ContainerDied","Data":"d878e5f2a1e518a5ab973daca1b339066f59dbd784d21bfb639d8f9eef5d5b89"}
Sep 09 21:38:24.341388 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:24.341357    2578 generic.go:356] "Generic (PLEG): container finished" podID="e6dec92b-3600-4d2b-a578-92b4382aca7d" containerID="89afb14935b88cdea05bc1a27d5eec03e3a9c4ed638df6fa3561b1ac1d78630f" exitCode=0
Sep 09 21:38:24.341804 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:24.341435    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7" event={"ID":"e6dec92b-3600-4d2b-a578-92b4382aca7d","Type":"ContainerDied","Data":"89afb14935b88cdea05bc1a27d5eec03e3a9c4ed638df6fa3561b1ac1d78630f"}
Sep 09 21:38:25.472709 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:25.472687    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"
Sep 09 21:38:25.597740 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:25.597665    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sg4dg\" (UniqueName: \"kubernetes.io/projected/e6dec92b-3600-4d2b-a578-92b4382aca7d-kube-api-access-sg4dg\") pod \"e6dec92b-3600-4d2b-a578-92b4382aca7d\" (UID: \"e6dec92b-3600-4d2b-a578-92b4382aca7d\") "
Sep 09 21:38:25.597740 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:25.597705    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6dec92b-3600-4d2b-a578-92b4382aca7d-util\") pod \"e6dec92b-3600-4d2b-a578-92b4382aca7d\" (UID: \"e6dec92b-3600-4d2b-a578-92b4382aca7d\") "
Sep 09 21:38:25.597740 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:25.597732    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6dec92b-3600-4d2b-a578-92b4382aca7d-bundle\") pod \"e6dec92b-3600-4d2b-a578-92b4382aca7d\" (UID: \"e6dec92b-3600-4d2b-a578-92b4382aca7d\") "
Sep 09 21:38:25.598518 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:25.598481    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6dec92b-3600-4d2b-a578-92b4382aca7d-bundle" (OuterVolumeSpecName: "bundle") pod "e6dec92b-3600-4d2b-a578-92b4382aca7d" (UID: "e6dec92b-3600-4d2b-a578-92b4382aca7d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:38:25.599729 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:25.599702    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6dec92b-3600-4d2b-a578-92b4382aca7d-kube-api-access-sg4dg" (OuterVolumeSpecName: "kube-api-access-sg4dg") pod "e6dec92b-3600-4d2b-a578-92b4382aca7d" (UID: "e6dec92b-3600-4d2b-a578-92b4382aca7d"). InnerVolumeSpecName "kube-api-access-sg4dg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:38:25.604676 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:25.604651    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6dec92b-3600-4d2b-a578-92b4382aca7d-util" (OuterVolumeSpecName: "util") pod "e6dec92b-3600-4d2b-a578-92b4382aca7d" (UID: "e6dec92b-3600-4d2b-a578-92b4382aca7d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:38:25.698703 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:25.698678    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sg4dg\" (UniqueName: \"kubernetes.io/projected/e6dec92b-3600-4d2b-a578-92b4382aca7d-kube-api-access-sg4dg\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:38:25.698703 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:25.698699    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6dec92b-3600-4d2b-a578-92b4382aca7d-util\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:38:25.698824 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:25.698710    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6dec92b-3600-4d2b-a578-92b4382aca7d-bundle\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:38:26.351188 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:26.351155    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj" event={"ID":"03067d81-ba5a-44ca-abe3-6c5f9f1e166c","Type":"ContainerStarted","Data":"f0c061d804e7e9937afa5a54bff2c0d922447e7f5c7cb1b5ba21ca7c382e5a61"}
Sep 09 21:38:26.353088 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:26.353067    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7"
Sep 09 21:38:26.353088 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:26.353076    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0m4nd7" event={"ID":"e6dec92b-3600-4d2b-a578-92b4382aca7d","Type":"ContainerDied","Data":"e3e007a449f49a6a8c10da4149380406a8305cecc195c2906bcb5d664a92b40e"}
Sep 09 21:38:26.353218 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:26.353110    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3e007a449f49a6a8c10da4149380406a8305cecc195c2906bcb5d664a92b40e"
Sep 09 21:38:27.358642 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:27.358609    2578 generic.go:356] "Generic (PLEG): container finished" podID="03067d81-ba5a-44ca-abe3-6c5f9f1e166c" containerID="f0c061d804e7e9937afa5a54bff2c0d922447e7f5c7cb1b5ba21ca7c382e5a61" exitCode=0
Sep 09 21:38:27.359036 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:27.358697    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj" event={"ID":"03067d81-ba5a-44ca-abe3-6c5f9f1e166c","Type":"ContainerDied","Data":"f0c061d804e7e9937afa5a54bff2c0d922447e7f5c7cb1b5ba21ca7c382e5a61"}
Sep 09 21:38:28.366511 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:28.366472    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj" event={"ID":"03067d81-ba5a-44ca-abe3-6c5f9f1e166c","Type":"ContainerStarted","Data":"58333867d4ca3ac038047f6b7d4d5ce2a37e323234e8aadbce7679e871a427e7"}
Sep 09 21:38:28.366930 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:28.366668    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:28.389090 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:28.389040    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj" podStartSLOduration=2.962914857 podStartE2EDuration="7.389024805s" totalImagesPullingTime="3.423115171s" totalInitContainerRuntime="1.002994777s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:38:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:38:28.38654069 +0000 UTC m=+327.859505621" watchObservedRunningTime="2026-09-09 21:38:28.389024805 +0000 UTC m=+327.861989716"
Sep 09 21:38:33.335026 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:33.334996    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-64d94cf4d-hjk2v"
Sep 09 21:38:39.372944 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:39.372912    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84fbbd4d59-wh2vj"
Sep 09 21:38:48.380024 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.379990    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"]
Sep 09 21:38:48.380365 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.380338    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6dec92b-3600-4d2b-a578-92b4382aca7d" containerName="util"
Sep 09 21:38:48.380365 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.380351    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6dec92b-3600-4d2b-a578-92b4382aca7d" containerName="util"
Sep 09 21:38:48.380455 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.380372    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6dec92b-3600-4d2b-a578-92b4382aca7d" containerName="extract"
Sep 09 21:38:48.380455 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.380381    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6dec92b-3600-4d2b-a578-92b4382aca7d" containerName="extract"
Sep 09 21:38:48.380455 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.380399    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6dec92b-3600-4d2b-a578-92b4382aca7d" containerName="pull"
Sep 09 21:38:48.380455 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.380405    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6dec92b-3600-4d2b-a578-92b4382aca7d" containerName="pull"
Sep 09 21:38:48.380568 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.380459    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6dec92b-3600-4d2b-a578-92b4382aca7d" containerName="extract"
Sep 09 21:38:48.388566 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.388544    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.391373 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.391352    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 21:38:48.391485 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.391399    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-vkcnn\""
Sep 09 21:38:48.396109 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.396016    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"]
Sep 09 21:38:48.466429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.466402    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3e588e7-7b28-45e8-a99b-a7b2467a5916-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.466545 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.466436    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3e588e7-7b28-45e8-a99b-a7b2467a5916-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.466545 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.466456    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3e588e7-7b28-45e8-a99b-a7b2467a5916-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.466545 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.466530    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3e588e7-7b28-45e8-a99b-a7b2467a5916-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.466652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.466552    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3e588e7-7b28-45e8-a99b-a7b2467a5916-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.466652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.466577    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr8hn\" (UniqueName: \"kubernetes.io/projected/d3e588e7-7b28-45e8-a99b-a7b2467a5916-kube-api-access-fr8hn\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.466652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.466625    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3e588e7-7b28-45e8-a99b-a7b2467a5916-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.466652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.466650    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3e588e7-7b28-45e8-a99b-a7b2467a5916-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.466801 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.466672    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3e588e7-7b28-45e8-a99b-a7b2467a5916-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.568046 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.568015    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3e588e7-7b28-45e8-a99b-a7b2467a5916-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.568046 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.568050    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3e588e7-7b28-45e8-a99b-a7b2467a5916-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.568283 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.568071    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3e588e7-7b28-45e8-a99b-a7b2467a5916-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.568283 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.568093    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3e588e7-7b28-45e8-a99b-a7b2467a5916-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.568283 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.568113    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3e588e7-7b28-45e8-a99b-a7b2467a5916-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.568283 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.568133    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3e588e7-7b28-45e8-a99b-a7b2467a5916-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.568283 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.568185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3e588e7-7b28-45e8-a99b-a7b2467a5916-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.568283 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.568210    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3e588e7-7b28-45e8-a99b-a7b2467a5916-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.568283 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.568242    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr8hn\" (UniqueName: \"kubernetes.io/projected/d3e588e7-7b28-45e8-a99b-a7b2467a5916-kube-api-access-fr8hn\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.568689 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.568508    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3e588e7-7b28-45e8-a99b-a7b2467a5916-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.568689 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.568550    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3e588e7-7b28-45e8-a99b-a7b2467a5916-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.568689 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.568632    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3e588e7-7b28-45e8-a99b-a7b2467a5916-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.568689 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.568682    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3e588e7-7b28-45e8-a99b-a7b2467a5916-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.568946 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.568923    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3e588e7-7b28-45e8-a99b-a7b2467a5916-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.570649 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.570630    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3e588e7-7b28-45e8-a99b-a7b2467a5916-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.570757 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.570736    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3e588e7-7b28-45e8-a99b-a7b2467a5916-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.581703 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.581677    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3e588e7-7b28-45e8-a99b-a7b2467a5916-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.583127 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.583104    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr8hn\" (UniqueName: \"kubernetes.io/projected/d3e588e7-7b28-45e8-a99b-a7b2467a5916-kube-api-access-fr8hn\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls\" (UID: \"d3e588e7-7b28-45e8-a99b-a7b2467a5916\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.699947 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.699885    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:48.978650 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:48.978627    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"]
Sep 09 21:38:48.980628 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:38:48.980595    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3e588e7_7b28_45e8_a99b_a7b2467a5916.slice/crio-f653d2c35b689879cbaec21e73d55511d99f72c04a8296f7feecbb4548cd8576 WatchSource:0}: Error finding container f653d2c35b689879cbaec21e73d55511d99f72c04a8296f7feecbb4548cd8576: Status 404 returned error can't find the container with id f653d2c35b689879cbaec21e73d55511d99f72c04a8296f7feecbb4548cd8576
Sep 09 21:38:49.463060 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:49.463023    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls" event={"ID":"d3e588e7-7b28-45e8-a99b-a7b2467a5916","Type":"ContainerStarted","Data":"f653d2c35b689879cbaec21e73d55511d99f72c04a8296f7feecbb4548cd8576"}
Sep 09 21:38:52.006988 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.006945    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-v8xm7"]
Sep 09 21:38:52.013757 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.013734    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-v8xm7"
Sep 09 21:38:52.017724 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.017686    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 09 21:38:52.017945 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.017925    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-6m5np\""
Sep 09 21:38:52.020098 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.020070    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-v8xm7"]
Sep 09 21:38:52.098122 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.098088    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05cf0125-a374-4a44-97ba-e9fdd79eee32-cert\") pod \"odh-model-controller-7998c67dc4-v8xm7\" (UID: \"05cf0125-a374-4a44-97ba-e9fdd79eee32\") " pod="opendatahub/odh-model-controller-7998c67dc4-v8xm7"
Sep 09 21:38:52.098358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.098279    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k29g\" (UniqueName: \"kubernetes.io/projected/05cf0125-a374-4a44-97ba-e9fdd79eee32-kube-api-access-5k29g\") pod \"odh-model-controller-7998c67dc4-v8xm7\" (UID: \"05cf0125-a374-4a44-97ba-e9fdd79eee32\") " pod="opendatahub/odh-model-controller-7998c67dc4-v8xm7"
Sep 09 21:38:52.199185 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.199151    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05cf0125-a374-4a44-97ba-e9fdd79eee32-cert\") pod \"odh-model-controller-7998c67dc4-v8xm7\" (UID: \"05cf0125-a374-4a44-97ba-e9fdd79eee32\") " pod="opendatahub/odh-model-controller-7998c67dc4-v8xm7"
Sep 09 21:38:52.199426 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.199238    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k29g\" (UniqueName: \"kubernetes.io/projected/05cf0125-a374-4a44-97ba-e9fdd79eee32-kube-api-access-5k29g\") pod \"odh-model-controller-7998c67dc4-v8xm7\" (UID: \"05cf0125-a374-4a44-97ba-e9fdd79eee32\") " pod="opendatahub/odh-model-controller-7998c67dc4-v8xm7"
Sep 09 21:38:52.201868 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.201834    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05cf0125-a374-4a44-97ba-e9fdd79eee32-cert\") pod \"odh-model-controller-7998c67dc4-v8xm7\" (UID: \"05cf0125-a374-4a44-97ba-e9fdd79eee32\") " pod="opendatahub/odh-model-controller-7998c67dc4-v8xm7"
Sep 09 21:38:52.207532 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.207502    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k29g\" (UniqueName: \"kubernetes.io/projected/05cf0125-a374-4a44-97ba-e9fdd79eee32-kube-api-access-5k29g\") pod \"odh-model-controller-7998c67dc4-v8xm7\" (UID: \"05cf0125-a374-4a44-97ba-e9fdd79eee32\") " pod="opendatahub/odh-model-controller-7998c67dc4-v8xm7"
Sep 09 21:38:52.328839 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.328802    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-v8xm7"
Sep 09 21:38:52.608890 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.608823    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-88srq"]
Sep 09 21:38:52.612028 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.612009    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-88srq"
Sep 09 21:38:52.614133 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.614111    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 09 21:38:52.614274 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.614198    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-c9sq7\""
Sep 09 21:38:52.645036 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.644945    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-88srq"]
Sep 09 21:38:52.669835 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.669203    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:38:52.669835 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.669302    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:38:52.669835 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.669398    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:38:52.704131 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.704106    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/533f307f-ef83-4756-9186-2aa44b9eacc6-tls-certs\") pod \"model-serving-api-54d89bd5fc-88srq\" (UID: \"533f307f-ef83-4756-9186-2aa44b9eacc6\") " pod="opendatahub/model-serving-api-54d89bd5fc-88srq"
Sep 09 21:38:52.704234 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.704193    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvpqk\" (UniqueName: \"kubernetes.io/projected/533f307f-ef83-4756-9186-2aa44b9eacc6-kube-api-access-vvpqk\") pod \"model-serving-api-54d89bd5fc-88srq\" (UID: \"533f307f-ef83-4756-9186-2aa44b9eacc6\") " pod="opendatahub/model-serving-api-54d89bd5fc-88srq"
Sep 09 21:38:52.804641 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.804603    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvpqk\" (UniqueName: \"kubernetes.io/projected/533f307f-ef83-4756-9186-2aa44b9eacc6-kube-api-access-vvpqk\") pod \"model-serving-api-54d89bd5fc-88srq\" (UID: \"533f307f-ef83-4756-9186-2aa44b9eacc6\") " pod="opendatahub/model-serving-api-54d89bd5fc-88srq"
Sep 09 21:38:52.804820 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.804671    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/533f307f-ef83-4756-9186-2aa44b9eacc6-tls-certs\") pod \"model-serving-api-54d89bd5fc-88srq\" (UID: \"533f307f-ef83-4756-9186-2aa44b9eacc6\") " pod="opendatahub/model-serving-api-54d89bd5fc-88srq"
Sep 09 21:38:52.804820 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:38:52.804782    2578 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 09 21:38:52.804949 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:38:52.804845    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/533f307f-ef83-4756-9186-2aa44b9eacc6-tls-certs podName:533f307f-ef83-4756-9186-2aa44b9eacc6 nodeName:}" failed. No retries permitted until 2026-09-09 21:38:53.304824875 +0000 UTC m=+352.777789768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/533f307f-ef83-4756-9186-2aa44b9eacc6-tls-certs") pod "model-serving-api-54d89bd5fc-88srq" (UID: "533f307f-ef83-4756-9186-2aa44b9eacc6") : secret "model-serving-api-tls" not found
Sep 09 21:38:52.812377 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.812351    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvpqk\" (UniqueName: \"kubernetes.io/projected/533f307f-ef83-4756-9186-2aa44b9eacc6-kube-api-access-vvpqk\") pod \"model-serving-api-54d89bd5fc-88srq\" (UID: \"533f307f-ef83-4756-9186-2aa44b9eacc6\") " pod="opendatahub/model-serving-api-54d89bd5fc-88srq"
Sep 09 21:38:52.848737 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:52.848686    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-v8xm7"]
Sep 09 21:38:52.852095 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:38:52.852064    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05cf0125_a374_4a44_97ba_e9fdd79eee32.slice/crio-891330b9ecb5ff5f3ec5a2ab1010ffd98862bd7301e4a54a6e14919117b5e17b WatchSource:0}: Error finding container 891330b9ecb5ff5f3ec5a2ab1010ffd98862bd7301e4a54a6e14919117b5e17b: Status 404 returned error can't find the container with id 891330b9ecb5ff5f3ec5a2ab1010ffd98862bd7301e4a54a6e14919117b5e17b
Sep 09 21:38:53.309137 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:53.309103    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/533f307f-ef83-4756-9186-2aa44b9eacc6-tls-certs\") pod \"model-serving-api-54d89bd5fc-88srq\" (UID: \"533f307f-ef83-4756-9186-2aa44b9eacc6\") " pod="opendatahub/model-serving-api-54d89bd5fc-88srq"
Sep 09 21:38:53.311692 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:53.311662    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/533f307f-ef83-4756-9186-2aa44b9eacc6-tls-certs\") pod \"model-serving-api-54d89bd5fc-88srq\" (UID: \"533f307f-ef83-4756-9186-2aa44b9eacc6\") " pod="opendatahub/model-serving-api-54d89bd5fc-88srq"
Sep 09 21:38:53.484721 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:53.484682    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-v8xm7" event={"ID":"05cf0125-a374-4a44-97ba-e9fdd79eee32","Type":"ContainerStarted","Data":"891330b9ecb5ff5f3ec5a2ab1010ffd98862bd7301e4a54a6e14919117b5e17b"}
Sep 09 21:38:53.486153 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:53.486125    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls" event={"ID":"d3e588e7-7b28-45e8-a99b-a7b2467a5916","Type":"ContainerStarted","Data":"eeb31a1ece8971e70547ab55f0455d2b08575c03a1088bd7ac3e8ca9bfee6cc4"}
Sep 09 21:38:53.503417 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:53.503372    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls" podStartSLOduration=1.817153094 podStartE2EDuration="5.503357925s" totalImagesPullingTime="3.686204831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:38:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:38:53.502396785 +0000 UTC m=+352.975361695" watchObservedRunningTime="2026-09-09 21:38:53.503357925 +0000 UTC m=+352.976322835"
Sep 09 21:38:53.541597 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:53.541570    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-88srq"
Sep 09 21:38:53.700521 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:53.700472    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:53.705705 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:53.705672    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:53.829442 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:53.829417    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-88srq"]
Sep 09 21:38:53.834378 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:38:53.834347    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod533f307f_ef83_4756_9186_2aa44b9eacc6.slice/crio-92b405e1d19c0af6bb2e6bdb21c43f863ad0b3b315d6d41e4a2eb789f7a95ebc WatchSource:0}: Error finding container 92b405e1d19c0af6bb2e6bdb21c43f863ad0b3b315d6d41e4a2eb789f7a95ebc: Status 404 returned error can't find the container with id 92b405e1d19c0af6bb2e6bdb21c43f863ad0b3b315d6d41e4a2eb789f7a95ebc
Sep 09 21:38:54.497595 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:54.497451    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-88srq" event={"ID":"533f307f-ef83-4756-9186-2aa44b9eacc6","Type":"ContainerStarted","Data":"92b405e1d19c0af6bb2e6bdb21c43f863ad0b3b315d6d41e4a2eb789f7a95ebc"}
Sep 09 21:38:54.498410 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:54.498389    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:54.499515 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:54.499494    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls"
Sep 09 21:38:58.525663 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:58.525626    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-v8xm7" event={"ID":"05cf0125-a374-4a44-97ba-e9fdd79eee32","Type":"ContainerStarted","Data":"eebc984204515c33abdf80adf72fe50a86a354d32ad313621fe0a5149d8afb70"}
Sep 09 21:38:58.526154 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:58.525763    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-v8xm7"
Sep 09 21:38:58.527354 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:58.527331    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-88srq" event={"ID":"533f307f-ef83-4756-9186-2aa44b9eacc6","Type":"ContainerStarted","Data":"ce792e6a5535e875fc5d19d27d830686addcae5bad7eaf1d2014921326c4f2fe"}
Sep 09 21:38:58.527528 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:58.527452    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-88srq"
Sep 09 21:38:58.543018 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:58.542958    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-v8xm7" podStartSLOduration=2.986131225 podStartE2EDuration="7.542945461s" totalImagesPullingTime="4.556814236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:38:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:38:58.541003196 +0000 UTC m=+358.013968105" watchObservedRunningTime="2026-09-09 21:38:58.542945461 +0000 UTC m=+358.015910402"
Sep 09 21:38:58.559027 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:38:58.558982    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-88srq" podStartSLOduration=2.943763808 podStartE2EDuration="6.558957312s" totalImagesPullingTime="3.615193504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:38:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:38:58.556781944 +0000 UTC m=+358.029746854" watchObservedRunningTime="2026-09-09 21:38:58.558957312 +0000 UTC m=+358.031922222"
Sep 09 21:39:01.938947 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:01.938915    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7l92p"]
Sep 09 21:39:01.943822 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:01.943802    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7l92p"
Sep 09 21:39:01.947337 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:01.947313    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 09 21:39:01.947465 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:01.947439    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4qqvh\""
Sep 09 21:39:01.956135 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:01.956113    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr"]
Sep 09 21:39:01.959297 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:01.959275    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7l92p"]
Sep 09 21:39:01.959401 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:01.959360    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr"
Sep 09 21:39:01.961894 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:01.961443    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 09 21:39:01.961894 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:01.961714    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-lwhxr\""
Sep 09 21:39:01.971217 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:01.971196    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr"]
Sep 09 21:39:02.085274 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.085246    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngxzv\" (UniqueName: \"kubernetes.io/projected/95e3f673-2fac-4251-82e0-433956f63519-kube-api-access-ngxzv\") pod \"llmisvc-controller-manager-777c6bd7d5-7tdjr\" (UID: \"95e3f673-2fac-4251-82e0-433956f63519\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr"
Sep 09 21:39:02.085414 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.085336    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4th8\" (UniqueName: \"kubernetes.io/projected/14b3dc4c-298c-49cd-abe5-bbd1c54a87fb-kube-api-access-d4th8\") pod \"kserve-controller-manager-d6f4b65b8-7l92p\" (UID: \"14b3dc4c-298c-49cd-abe5-bbd1c54a87fb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7l92p"
Sep 09 21:39:02.085414 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.085375    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95e3f673-2fac-4251-82e0-433956f63519-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7tdjr\" (UID: \"95e3f673-2fac-4251-82e0-433956f63519\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr"
Sep 09 21:39:02.085414 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.085410    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14b3dc4c-298c-49cd-abe5-bbd1c54a87fb-cert\") pod \"kserve-controller-manager-d6f4b65b8-7l92p\" (UID: \"14b3dc4c-298c-49cd-abe5-bbd1c54a87fb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7l92p"
Sep 09 21:39:02.186357 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.186328    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95e3f673-2fac-4251-82e0-433956f63519-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7tdjr\" (UID: \"95e3f673-2fac-4251-82e0-433956f63519\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr"
Sep 09 21:39:02.186498 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.186372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14b3dc4c-298c-49cd-abe5-bbd1c54a87fb-cert\") pod \"kserve-controller-manager-d6f4b65b8-7l92p\" (UID: \"14b3dc4c-298c-49cd-abe5-bbd1c54a87fb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7l92p"
Sep 09 21:39:02.186498 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.186409    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngxzv\" (UniqueName: \"kubernetes.io/projected/95e3f673-2fac-4251-82e0-433956f63519-kube-api-access-ngxzv\") pod \"llmisvc-controller-manager-777c6bd7d5-7tdjr\" (UID: \"95e3f673-2fac-4251-82e0-433956f63519\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr"
Sep 09 21:39:02.186498 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.186444    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4th8\" (UniqueName: \"kubernetes.io/projected/14b3dc4c-298c-49cd-abe5-bbd1c54a87fb-kube-api-access-d4th8\") pod \"kserve-controller-manager-d6f4b65b8-7l92p\" (UID: \"14b3dc4c-298c-49cd-abe5-bbd1c54a87fb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7l92p"
Sep 09 21:39:02.186677 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:39:02.186544    2578 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 09 21:39:02.186677 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:39:02.186612    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14b3dc4c-298c-49cd-abe5-bbd1c54a87fb-cert podName:14b3dc4c-298c-49cd-abe5-bbd1c54a87fb nodeName:}" failed. No retries permitted until 2026-09-09 21:39:02.686597153 +0000 UTC m=+362.159562041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/14b3dc4c-298c-49cd-abe5-bbd1c54a87fb-cert") pod "kserve-controller-manager-d6f4b65b8-7l92p" (UID: "14b3dc4c-298c-49cd-abe5-bbd1c54a87fb") : secret "kserve-webhook-server-cert" not found
Sep 09 21:39:02.188860 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.188835    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95e3f673-2fac-4251-82e0-433956f63519-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7tdjr\" (UID: \"95e3f673-2fac-4251-82e0-433956f63519\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr"
Sep 09 21:39:02.200548 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.200493    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4th8\" (UniqueName: \"kubernetes.io/projected/14b3dc4c-298c-49cd-abe5-bbd1c54a87fb-kube-api-access-d4th8\") pod \"kserve-controller-manager-d6f4b65b8-7l92p\" (UID: \"14b3dc4c-298c-49cd-abe5-bbd1c54a87fb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7l92p"
Sep 09 21:39:02.201303 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.201277    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngxzv\" (UniqueName: \"kubernetes.io/projected/95e3f673-2fac-4251-82e0-433956f63519-kube-api-access-ngxzv\") pod \"llmisvc-controller-manager-777c6bd7d5-7tdjr\" (UID: \"95e3f673-2fac-4251-82e0-433956f63519\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr"
Sep 09 21:39:02.270673 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.270649    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr"
Sep 09 21:39:02.538987 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.538941    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr"]
Sep 09 21:39:02.542274 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:39:02.542246    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod95e3f673_2fac_4251_82e0_433956f63519.slice/crio-2a868b4d6ba40befc8073e9e50c97574d55ca056db3a9fd346c4890764e7274c WatchSource:0}: Error finding container 2a868b4d6ba40befc8073e9e50c97574d55ca056db3a9fd346c4890764e7274c: Status 404 returned error can't find the container with id 2a868b4d6ba40befc8073e9e50c97574d55ca056db3a9fd346c4890764e7274c
Sep 09 21:39:02.689951 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.689919    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14b3dc4c-298c-49cd-abe5-bbd1c54a87fb-cert\") pod \"kserve-controller-manager-d6f4b65b8-7l92p\" (UID: \"14b3dc4c-298c-49cd-abe5-bbd1c54a87fb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7l92p"
Sep 09 21:39:02.692213 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.692195    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14b3dc4c-298c-49cd-abe5-bbd1c54a87fb-cert\") pod \"kserve-controller-manager-d6f4b65b8-7l92p\" (UID: \"14b3dc4c-298c-49cd-abe5-bbd1c54a87fb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7l92p"
Sep 09 21:39:02.853794 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:02.853724    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7l92p"
Sep 09 21:39:03.130416 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:03.130392    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7l92p"]
Sep 09 21:39:03.132553 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:39:03.132520    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14b3dc4c_298c_49cd_abe5_bbd1c54a87fb.slice/crio-0270be46c88045bfded531d5df9fd6caf29d61a09ac350955b4b2807c592c357 WatchSource:0}: Error finding container 0270be46c88045bfded531d5df9fd6caf29d61a09ac350955b4b2807c592c357: Status 404 returned error can't find the container with id 0270be46c88045bfded531d5df9fd6caf29d61a09ac350955b4b2807c592c357
Sep 09 21:39:03.551439 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:03.551407    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7l92p" event={"ID":"14b3dc4c-298c-49cd-abe5-bbd1c54a87fb","Type":"ContainerStarted","Data":"0270be46c88045bfded531d5df9fd6caf29d61a09ac350955b4b2807c592c357"}
Sep 09 21:39:03.552555 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:03.552531    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr" event={"ID":"95e3f673-2fac-4251-82e0-433956f63519","Type":"ContainerStarted","Data":"2a868b4d6ba40befc8073e9e50c97574d55ca056db3a9fd346c4890764e7274c"}
Sep 09 21:39:06.570284 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:06.570252    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7l92p" event={"ID":"14b3dc4c-298c-49cd-abe5-bbd1c54a87fb","Type":"ContainerStarted","Data":"fb8171c8f6aefbd5c8960d856e56764b65ac627bdecbf9c614ce4cb0696f19c9"}
Sep 09 21:39:06.570620 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:06.570415    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7l92p"
Sep 09 21:39:06.571774 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:06.571752    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr" event={"ID":"95e3f673-2fac-4251-82e0-433956f63519","Type":"ContainerStarted","Data":"8a3b8c5a4aa85521921539d676fa1f4ec4a8a82ef9a4b0b59f400f9aa776b8e3"}
Sep 09 21:39:06.571863 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:06.571803    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr"
Sep 09 21:39:06.588595 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:06.588553    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7l92p" podStartSLOduration=2.984032057 podStartE2EDuration="5.588538518s" totalImagesPullingTime="2.604506461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:39:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:39:06.586426975 +0000 UTC m=+366.059391892" watchObservedRunningTime="2026-09-09 21:39:06.588538518 +0000 UTC m=+366.061503429"
Sep 09 21:39:06.603363 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:06.603321    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr" podStartSLOduration=1.6772148059999998 podStartE2EDuration="5.603307871s" totalImagesPullingTime="3.926093065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:39:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:39:06.601474449 +0000 UTC m=+366.074439358" watchObservedRunningTime="2026-09-09 21:39:06.603307871 +0000 UTC m=+366.076272826"
Sep 09 21:39:09.534078 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:09.534043    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-v8xm7"
Sep 09 21:39:09.536775 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:09.536751    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-88srq"
Sep 09 21:39:37.579253 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:37.579220    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7tdjr"
Sep 09 21:39:37.582694 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:37.582669    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7l92p"
Sep 09 21:39:56.788410 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:56.788373    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"]
Sep 09 21:39:56.791828 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:56.791807    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"
Sep 09 21:39:56.794280 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:56.794257    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 21:39:56.795415 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:56.795388    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 21:39:56.795528 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:56.795414    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9cxbc\""
Sep 09 21:39:56.801121 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:56.801096    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"]
Sep 09 21:39:56.890113 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:56.890085    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"]
Sep 09 21:39:56.893378 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:56.893363    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"
Sep 09 21:39:56.902681 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:56.902659    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"]
Sep 09 21:39:56.918015 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:56.917994    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/441de817-8378-4459-9103-660e9fcc735f-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9\" (UID: \"441de817-8378-4459-9103-660e9fcc735f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"
Sep 09 21:39:56.918104 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:56.918040    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/441de817-8378-4459-9103-660e9fcc735f-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9\" (UID: \"441de817-8378-4459-9103-660e9fcc735f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"
Sep 09 21:39:56.918104 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:56.918064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w85h8\" (UniqueName: \"kubernetes.io/projected/441de817-8378-4459-9103-660e9fcc735f-kube-api-access-w85h8\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9\" (UID: \"441de817-8378-4459-9103-660e9fcc735f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"
Sep 09 21:39:56.990192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:56.990167    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"]
Sep 09 21:39:56.993457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:56.993441    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"
Sep 09 21:39:57.004286 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.004267    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"]
Sep 09 21:39:57.018425 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.018403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4eba010d-604c-4915-bc95-1d51ee05faf0-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47\" (UID: \"4eba010d-604c-4915-bc95-1d51ee05faf0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"
Sep 09 21:39:57.018508 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.018438    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/441de817-8378-4459-9103-660e9fcc735f-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9\" (UID: \"441de817-8378-4459-9103-660e9fcc735f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"
Sep 09 21:39:57.018508 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.018459    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w85h8\" (UniqueName: \"kubernetes.io/projected/441de817-8378-4459-9103-660e9fcc735f-kube-api-access-w85h8\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9\" (UID: \"441de817-8378-4459-9103-660e9fcc735f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"
Sep 09 21:39:57.018578 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.018506    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hcqr\" (UniqueName: \"kubernetes.io/projected/4eba010d-604c-4915-bc95-1d51ee05faf0-kube-api-access-8hcqr\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47\" (UID: \"4eba010d-604c-4915-bc95-1d51ee05faf0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"
Sep 09 21:39:57.018617 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.018595    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4eba010d-604c-4915-bc95-1d51ee05faf0-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47\" (UID: \"4eba010d-604c-4915-bc95-1d51ee05faf0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"
Sep 09 21:39:57.018654 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.018628    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/441de817-8378-4459-9103-660e9fcc735f-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9\" (UID: \"441de817-8378-4459-9103-660e9fcc735f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"
Sep 09 21:39:57.018793 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.018776    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/441de817-8378-4459-9103-660e9fcc735f-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9\" (UID: \"441de817-8378-4459-9103-660e9fcc735f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"
Sep 09 21:39:57.018903 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.018885    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/441de817-8378-4459-9103-660e9fcc735f-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9\" (UID: \"441de817-8378-4459-9103-660e9fcc735f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"
Sep 09 21:39:57.026262 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.026240    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w85h8\" (UniqueName: \"kubernetes.io/projected/441de817-8378-4459-9103-660e9fcc735f-kube-api-access-w85h8\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9\" (UID: \"441de817-8378-4459-9103-660e9fcc735f\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"
Sep 09 21:39:57.091196 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.091142    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"]
Sep 09 21:39:57.094738 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.094722    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"
Sep 09 21:39:57.102167 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.102133    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"]
Sep 09 21:39:57.102432 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.102412    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"
Sep 09 21:39:57.119639 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.119610    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4eba010d-604c-4915-bc95-1d51ee05faf0-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47\" (UID: \"4eba010d-604c-4915-bc95-1d51ee05faf0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"
Sep 09 21:39:57.119742 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.119684    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hcqr\" (UniqueName: \"kubernetes.io/projected/4eba010d-604c-4915-bc95-1d51ee05faf0-kube-api-access-8hcqr\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47\" (UID: \"4eba010d-604c-4915-bc95-1d51ee05faf0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"
Sep 09 21:39:57.119806 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.119759    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f86c296b-b555-4f4a-8d88-d24967eed2bb-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp\" (UID: \"f86c296b-b555-4f4a-8d88-d24967eed2bb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"
Sep 09 21:39:57.119871 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.119803    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f86c296b-b555-4f4a-8d88-d24967eed2bb-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp\" (UID: \"f86c296b-b555-4f4a-8d88-d24967eed2bb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"
Sep 09 21:39:57.119935 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.119917    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4eba010d-604c-4915-bc95-1d51ee05faf0-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47\" (UID: \"4eba010d-604c-4915-bc95-1d51ee05faf0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"
Sep 09 21:39:57.120022 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.119955    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zw5b\" (UniqueName: \"kubernetes.io/projected/f86c296b-b555-4f4a-8d88-d24967eed2bb-kube-api-access-8zw5b\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp\" (UID: \"f86c296b-b555-4f4a-8d88-d24967eed2bb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"
Sep 09 21:39:57.120085 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.120041    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4eba010d-604c-4915-bc95-1d51ee05faf0-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47\" (UID: \"4eba010d-604c-4915-bc95-1d51ee05faf0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"
Sep 09 21:39:57.120352 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.120329    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4eba010d-604c-4915-bc95-1d51ee05faf0-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47\" (UID: \"4eba010d-604c-4915-bc95-1d51ee05faf0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"
Sep 09 21:39:57.127413 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.127388    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hcqr\" (UniqueName: \"kubernetes.io/projected/4eba010d-604c-4915-bc95-1d51ee05faf0-kube-api-access-8hcqr\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47\" (UID: \"4eba010d-604c-4915-bc95-1d51ee05faf0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"
Sep 09 21:39:57.202019 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.201992    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"
Sep 09 21:39:57.221267 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.221242    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knkmk\" (UniqueName: \"kubernetes.io/projected/d0446eae-86a6-4568-b754-d4ce715e75ec-kube-api-access-knkmk\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq\" (UID: \"d0446eae-86a6-4568-b754-d4ce715e75ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"
Sep 09 21:39:57.221385 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.221305    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0446eae-86a6-4568-b754-d4ce715e75ec-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq\" (UID: \"d0446eae-86a6-4568-b754-d4ce715e75ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"
Sep 09 21:39:57.221475 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.221450    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f86c296b-b555-4f4a-8d88-d24967eed2bb-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp\" (UID: \"f86c296b-b555-4f4a-8d88-d24967eed2bb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"
Sep 09 21:39:57.221528 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.221505    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f86c296b-b555-4f4a-8d88-d24967eed2bb-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp\" (UID: \"f86c296b-b555-4f4a-8d88-d24967eed2bb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"
Sep 09 21:39:57.221581 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.221546    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0446eae-86a6-4568-b754-d4ce715e75ec-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq\" (UID: \"d0446eae-86a6-4568-b754-d4ce715e75ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"
Sep 09 21:39:57.221627 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.221581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zw5b\" (UniqueName: \"kubernetes.io/projected/f86c296b-b555-4f4a-8d88-d24967eed2bb-kube-api-access-8zw5b\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp\" (UID: \"f86c296b-b555-4f4a-8d88-d24967eed2bb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"
Sep 09 21:39:57.221893 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.221867    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f86c296b-b555-4f4a-8d88-d24967eed2bb-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp\" (UID: \"f86c296b-b555-4f4a-8d88-d24967eed2bb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"
Sep 09 21:39:57.221959 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.221871    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f86c296b-b555-4f4a-8d88-d24967eed2bb-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp\" (UID: \"f86c296b-b555-4f4a-8d88-d24967eed2bb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"
Sep 09 21:39:57.229433 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.229410    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zw5b\" (UniqueName: \"kubernetes.io/projected/f86c296b-b555-4f4a-8d88-d24967eed2bb-kube-api-access-8zw5b\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp\" (UID: \"f86c296b-b555-4f4a-8d88-d24967eed2bb\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"
Sep 09 21:39:57.302938 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.302912    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"
Sep 09 21:39:57.322176 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.322153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0446eae-86a6-4568-b754-d4ce715e75ec-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq\" (UID: \"d0446eae-86a6-4568-b754-d4ce715e75ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"
Sep 09 21:39:57.322290 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.322208    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0446eae-86a6-4568-b754-d4ce715e75ec-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq\" (UID: \"d0446eae-86a6-4568-b754-d4ce715e75ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"
Sep 09 21:39:57.322290 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.322250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knkmk\" (UniqueName: \"kubernetes.io/projected/d0446eae-86a6-4568-b754-d4ce715e75ec-kube-api-access-knkmk\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq\" (UID: \"d0446eae-86a6-4568-b754-d4ce715e75ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"
Sep 09 21:39:57.322516 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.322495    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0446eae-86a6-4568-b754-d4ce715e75ec-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq\" (UID: \"d0446eae-86a6-4568-b754-d4ce715e75ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"
Sep 09 21:39:57.322622 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.322601    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0446eae-86a6-4568-b754-d4ce715e75ec-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq\" (UID: \"d0446eae-86a6-4568-b754-d4ce715e75ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"
Sep 09 21:39:57.330163 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.330144    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knkmk\" (UniqueName: \"kubernetes.io/projected/d0446eae-86a6-4568-b754-d4ce715e75ec-kube-api-access-knkmk\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq\" (UID: \"d0446eae-86a6-4568-b754-d4ce715e75ec\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"
Sep 09 21:39:57.374774 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.374748    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"]
Sep 09 21:39:57.377724 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:39:57.377700    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod441de817_8378_4459_9103_660e9fcc735f.slice/crio-525840d99f9787e9c53abf7183ad18e08fb3c85bd890bba48fccf2918d25afcd WatchSource:0}: Error finding container 525840d99f9787e9c53abf7183ad18e08fb3c85bd890bba48fccf2918d25afcd: Status 404 returned error can't find the container with id 525840d99f9787e9c53abf7183ad18e08fb3c85bd890bba48fccf2918d25afcd
Sep 09 21:39:57.405169 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.405148    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"
Sep 09 21:39:57.489044 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.488225    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"]
Sep 09 21:39:57.490326 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:39:57.489523    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eba010d_604c_4915_bc95_1d51ee05faf0.slice/crio-5916a66ab2670cf7b042c849267cd1354d696c19dbee28c273c18a209a8366a9 WatchSource:0}: Error finding container 5916a66ab2670cf7b042c849267cd1354d696c19dbee28c273c18a209a8366a9: Status 404 returned error can't find the container with id 5916a66ab2670cf7b042c849267cd1354d696c19dbee28c273c18a209a8366a9
Sep 09 21:39:57.573943 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.573913    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"]
Sep 09 21:39:57.575399 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:39:57.575374    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf86c296b_b555_4f4a_8d88_d24967eed2bb.slice/crio-e93733c0d884e46dd160d3059f0747793a0847b51856e085b127cbc24c69b284 WatchSource:0}: Error finding container e93733c0d884e46dd160d3059f0747793a0847b51856e085b127cbc24c69b284: Status 404 returned error can't find the container with id e93733c0d884e46dd160d3059f0747793a0847b51856e085b127cbc24c69b284
Sep 09 21:39:57.679490 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.679463    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"]
Sep 09 21:39:57.680867 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:39:57.680842    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0446eae_86a6_4568_b754_d4ce715e75ec.slice/crio-aef4a9edae2da4fc46bd79449a6c55b96a95d4d32139283d8f56fbfe8ccbde10 WatchSource:0}: Error finding container aef4a9edae2da4fc46bd79449a6c55b96a95d4d32139283d8f56fbfe8ccbde10: Status 404 returned error can't find the container with id aef4a9edae2da4fc46bd79449a6c55b96a95d4d32139283d8f56fbfe8ccbde10
Sep 09 21:39:57.816194 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.816163    2578 generic.go:356] "Generic (PLEG): container finished" podID="d0446eae-86a6-4568-b754-d4ce715e75ec" containerID="7823ee0d42d8111c3209562192b5ae4b6b1f2317ff0c3f200b919dbbac21de78" exitCode=0
Sep 09 21:39:57.816598 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.816228    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq" event={"ID":"d0446eae-86a6-4568-b754-d4ce715e75ec","Type":"ContainerDied","Data":"7823ee0d42d8111c3209562192b5ae4b6b1f2317ff0c3f200b919dbbac21de78"}
Sep 09 21:39:57.816598 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.816279    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq" event={"ID":"d0446eae-86a6-4568-b754-d4ce715e75ec","Type":"ContainerStarted","Data":"aef4a9edae2da4fc46bd79449a6c55b96a95d4d32139283d8f56fbfe8ccbde10"}
Sep 09 21:39:57.817878 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.817859    2578 generic.go:356] "Generic (PLEG): container finished" podID="4eba010d-604c-4915-bc95-1d51ee05faf0" containerID="d16fdc51cae96c3c3b684f2ef9aa89d5a7656bb46453121def53d92cf949d319" exitCode=0
Sep 09 21:39:57.817995 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.817955    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47" event={"ID":"4eba010d-604c-4915-bc95-1d51ee05faf0","Type":"ContainerDied","Data":"d16fdc51cae96c3c3b684f2ef9aa89d5a7656bb46453121def53d92cf949d319"}
Sep 09 21:39:57.818071 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.818018    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47" event={"ID":"4eba010d-604c-4915-bc95-1d51ee05faf0","Type":"ContainerStarted","Data":"5916a66ab2670cf7b042c849267cd1354d696c19dbee28c273c18a209a8366a9"}
Sep 09 21:39:57.819578 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.819556    2578 generic.go:356] "Generic (PLEG): container finished" podID="441de817-8378-4459-9103-660e9fcc735f" containerID="8b07db17184c35cbdaaafd3a6f20c8a409008d9a09a50353238ceed77b56db63" exitCode=0
Sep 09 21:39:57.819669 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.819630    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9" event={"ID":"441de817-8378-4459-9103-660e9fcc735f","Type":"ContainerDied","Data":"8b07db17184c35cbdaaafd3a6f20c8a409008d9a09a50353238ceed77b56db63"}
Sep 09 21:39:57.819724 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.819676    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9" event={"ID":"441de817-8378-4459-9103-660e9fcc735f","Type":"ContainerStarted","Data":"525840d99f9787e9c53abf7183ad18e08fb3c85bd890bba48fccf2918d25afcd"}
Sep 09 21:39:57.821226 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.821208    2578 generic.go:356] "Generic (PLEG): container finished" podID="f86c296b-b555-4f4a-8d88-d24967eed2bb" containerID="baf8b6f93d7315a8cd7c102517823c8cac763abdcdd040a071b790b2babff55f" exitCode=0
Sep 09 21:39:57.821313 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.821278    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp" event={"ID":"f86c296b-b555-4f4a-8d88-d24967eed2bb","Type":"ContainerDied","Data":"baf8b6f93d7315a8cd7c102517823c8cac763abdcdd040a071b790b2babff55f"}
Sep 09 21:39:57.821313 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:57.821307    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp" event={"ID":"f86c296b-b555-4f4a-8d88-d24967eed2bb","Type":"ContainerStarted","Data":"e93733c0d884e46dd160d3059f0747793a0847b51856e085b127cbc24c69b284"}
Sep 09 21:39:58.828096 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:58.828065    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47" event={"ID":"4eba010d-604c-4915-bc95-1d51ee05faf0","Type":"ContainerStarted","Data":"365f39313e43d0ab6b6c601b417d838d75c2bc869a35d9857b4c3d166daf7e73"}
Sep 09 21:39:59.834917 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:59.834884    2578 generic.go:356] "Generic (PLEG): container finished" podID="4eba010d-604c-4915-bc95-1d51ee05faf0" containerID="365f39313e43d0ab6b6c601b417d838d75c2bc869a35d9857b4c3d166daf7e73" exitCode=0
Sep 09 21:39:59.835351 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:59.834978    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47" event={"ID":"4eba010d-604c-4915-bc95-1d51ee05faf0","Type":"ContainerDied","Data":"365f39313e43d0ab6b6c601b417d838d75c2bc869a35d9857b4c3d166daf7e73"}
Sep 09 21:39:59.836865 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:59.836844    2578 generic.go:356] "Generic (PLEG): container finished" podID="441de817-8378-4459-9103-660e9fcc735f" containerID="af4e17b3cf303e1dd728be5d4d72d2d0310fdde82c428dd16ba944593d3230a7" exitCode=0
Sep 09 21:39:59.836936 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:59.836928    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9" event={"ID":"441de817-8378-4459-9103-660e9fcc735f","Type":"ContainerDied","Data":"af4e17b3cf303e1dd728be5d4d72d2d0310fdde82c428dd16ba944593d3230a7"}
Sep 09 21:39:59.838903 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:59.838882    2578 generic.go:356] "Generic (PLEG): container finished" podID="f86c296b-b555-4f4a-8d88-d24967eed2bb" containerID="ace2a50d97b157c6ed71690eb0096dd531495f614812d4627c264c41e27c11c8" exitCode=0
Sep 09 21:39:59.839013 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:59.838976    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp" event={"ID":"f86c296b-b555-4f4a-8d88-d24967eed2bb","Type":"ContainerDied","Data":"ace2a50d97b157c6ed71690eb0096dd531495f614812d4627c264c41e27c11c8"}
Sep 09 21:39:59.840930 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:59.840906    2578 generic.go:356] "Generic (PLEG): container finished" podID="d0446eae-86a6-4568-b754-d4ce715e75ec" containerID="38fd00cbed22c2c5f32bcb902ba135d68e7260d6b426eac0ab1affd50d1b8f9f" exitCode=0
Sep 09 21:39:59.841037 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:39:59.840939    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq" event={"ID":"d0446eae-86a6-4568-b754-d4ce715e75ec","Type":"ContainerDied","Data":"38fd00cbed22c2c5f32bcb902ba135d68e7260d6b426eac0ab1affd50d1b8f9f"}
Sep 09 21:40:00.848285 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:00.848244    2578 generic.go:356] "Generic (PLEG): container finished" podID="d0446eae-86a6-4568-b754-d4ce715e75ec" containerID="9f5678312934034b541003ce76e8ec419c5f5b0869d96c68cb12d17d51310545" exitCode=0
Sep 09 21:40:00.848736 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:00.848334    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq" event={"ID":"d0446eae-86a6-4568-b754-d4ce715e75ec","Type":"ContainerDied","Data":"9f5678312934034b541003ce76e8ec419c5f5b0869d96c68cb12d17d51310545"}
Sep 09 21:40:00.850521 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:00.850498    2578 generic.go:356] "Generic (PLEG): container finished" podID="4eba010d-604c-4915-bc95-1d51ee05faf0" containerID="55e639a32e73cee6a1fa6addd629577db802767927ab3eebefc77867cde30f8f" exitCode=0
Sep 09 21:40:00.850613 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:00.850587    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47" event={"ID":"4eba010d-604c-4915-bc95-1d51ee05faf0","Type":"ContainerDied","Data":"55e639a32e73cee6a1fa6addd629577db802767927ab3eebefc77867cde30f8f"}
Sep 09 21:40:00.852691 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:00.852671    2578 generic.go:356] "Generic (PLEG): container finished" podID="441de817-8378-4459-9103-660e9fcc735f" containerID="d613e7c6339b2a6b7c4553ddbd05bde3a829a12319d12b5883256668a7ce257f" exitCode=0
Sep 09 21:40:00.852791 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:00.852748    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9" event={"ID":"441de817-8378-4459-9103-660e9fcc735f","Type":"ContainerDied","Data":"d613e7c6339b2a6b7c4553ddbd05bde3a829a12319d12b5883256668a7ce257f"}
Sep 09 21:40:00.854695 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:00.854674    2578 generic.go:356] "Generic (PLEG): container finished" podID="f86c296b-b555-4f4a-8d88-d24967eed2bb" containerID="69ae503107a73e2a3f57577596e491a09a2b4cc6874f0fefa1e37e254aed7359" exitCode=0
Sep 09 21:40:00.854764 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:00.854721    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp" event={"ID":"f86c296b-b555-4f4a-8d88-d24967eed2bb","Type":"ContainerDied","Data":"69ae503107a73e2a3f57577596e491a09a2b4cc6874f0fefa1e37e254aed7359"}
Sep 09 21:40:02.024091 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.024067    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"
Sep 09 21:40:02.065398 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.065373    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"
Sep 09 21:40:02.095595 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.095571    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"
Sep 09 21:40:02.098660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.098639    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"
Sep 09 21:40:02.164621 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.164526    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hcqr\" (UniqueName: \"kubernetes.io/projected/4eba010d-604c-4915-bc95-1d51ee05faf0-kube-api-access-8hcqr\") pod \"4eba010d-604c-4915-bc95-1d51ee05faf0\" (UID: \"4eba010d-604c-4915-bc95-1d51ee05faf0\") "
Sep 09 21:40:02.164621 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.164565    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4eba010d-604c-4915-bc95-1d51ee05faf0-util\") pod \"4eba010d-604c-4915-bc95-1d51ee05faf0\" (UID: \"4eba010d-604c-4915-bc95-1d51ee05faf0\") "
Sep 09 21:40:02.164621 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.164601    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w85h8\" (UniqueName: \"kubernetes.io/projected/441de817-8378-4459-9103-660e9fcc735f-kube-api-access-w85h8\") pod \"441de817-8378-4459-9103-660e9fcc735f\" (UID: \"441de817-8378-4459-9103-660e9fcc735f\") "
Sep 09 21:40:02.164903 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.164628    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knkmk\" (UniqueName: \"kubernetes.io/projected/d0446eae-86a6-4568-b754-d4ce715e75ec-kube-api-access-knkmk\") pod \"d0446eae-86a6-4568-b754-d4ce715e75ec\" (UID: \"d0446eae-86a6-4568-b754-d4ce715e75ec\") "
Sep 09 21:40:02.164903 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.164663    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/441de817-8378-4459-9103-660e9fcc735f-bundle\") pod \"441de817-8378-4459-9103-660e9fcc735f\" (UID: \"441de817-8378-4459-9103-660e9fcc735f\") "
Sep 09 21:40:02.164903 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.164681    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4eba010d-604c-4915-bc95-1d51ee05faf0-bundle\") pod \"4eba010d-604c-4915-bc95-1d51ee05faf0\" (UID: \"4eba010d-604c-4915-bc95-1d51ee05faf0\") "
Sep 09 21:40:02.164903 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.164757    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0446eae-86a6-4568-b754-d4ce715e75ec-bundle\") pod \"d0446eae-86a6-4568-b754-d4ce715e75ec\" (UID: \"d0446eae-86a6-4568-b754-d4ce715e75ec\") "
Sep 09 21:40:02.164903 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.164792    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/441de817-8378-4459-9103-660e9fcc735f-util\") pod \"441de817-8378-4459-9103-660e9fcc735f\" (UID: \"441de817-8378-4459-9103-660e9fcc735f\") "
Sep 09 21:40:02.164903 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.164820    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0446eae-86a6-4568-b754-d4ce715e75ec-util\") pod \"d0446eae-86a6-4568-b754-d4ce715e75ec\" (UID: \"d0446eae-86a6-4568-b754-d4ce715e75ec\") "
Sep 09 21:40:02.165846 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.165286    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/441de817-8378-4459-9103-660e9fcc735f-bundle" (OuterVolumeSpecName: "bundle") pod "441de817-8378-4459-9103-660e9fcc735f" (UID: "441de817-8378-4459-9103-660e9fcc735f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:40:02.165846 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.165326    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eba010d-604c-4915-bc95-1d51ee05faf0-bundle" (OuterVolumeSpecName: "bundle") pod "4eba010d-604c-4915-bc95-1d51ee05faf0" (UID: "4eba010d-604c-4915-bc95-1d51ee05faf0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:40:02.166252 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.166217    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0446eae-86a6-4568-b754-d4ce715e75ec-bundle" (OuterVolumeSpecName: "bundle") pod "d0446eae-86a6-4568-b754-d4ce715e75ec" (UID: "d0446eae-86a6-4568-b754-d4ce715e75ec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:40:02.167316 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.167290    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eba010d-604c-4915-bc95-1d51ee05faf0-kube-api-access-8hcqr" (OuterVolumeSpecName: "kube-api-access-8hcqr") pod "4eba010d-604c-4915-bc95-1d51ee05faf0" (UID: "4eba010d-604c-4915-bc95-1d51ee05faf0"). InnerVolumeSpecName "kube-api-access-8hcqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:40:02.167432 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.167324    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0446eae-86a6-4568-b754-d4ce715e75ec-kube-api-access-knkmk" (OuterVolumeSpecName: "kube-api-access-knkmk") pod "d0446eae-86a6-4568-b754-d4ce715e75ec" (UID: "d0446eae-86a6-4568-b754-d4ce715e75ec"). InnerVolumeSpecName "kube-api-access-knkmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:40:02.168989 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.168943    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/441de817-8378-4459-9103-660e9fcc735f-kube-api-access-w85h8" (OuterVolumeSpecName: "kube-api-access-w85h8") pod "441de817-8378-4459-9103-660e9fcc735f" (UID: "441de817-8378-4459-9103-660e9fcc735f"). InnerVolumeSpecName "kube-api-access-w85h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:40:02.171459 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.171436    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eba010d-604c-4915-bc95-1d51ee05faf0-util" (OuterVolumeSpecName: "util") pod "4eba010d-604c-4915-bc95-1d51ee05faf0" (UID: "4eba010d-604c-4915-bc95-1d51ee05faf0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:40:02.171753 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.171735    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/441de817-8378-4459-9103-660e9fcc735f-util" (OuterVolumeSpecName: "util") pod "441de817-8378-4459-9103-660e9fcc735f" (UID: "441de817-8378-4459-9103-660e9fcc735f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:40:02.171904 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.171884    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0446eae-86a6-4568-b754-d4ce715e75ec-util" (OuterVolumeSpecName: "util") pod "d0446eae-86a6-4568-b754-d4ce715e75ec" (UID: "d0446eae-86a6-4568-b754-d4ce715e75ec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:40:02.265991 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.265941    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f86c296b-b555-4f4a-8d88-d24967eed2bb-util\") pod \"f86c296b-b555-4f4a-8d88-d24967eed2bb\" (UID: \"f86c296b-b555-4f4a-8d88-d24967eed2bb\") "
Sep 09 21:40:02.266144 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.266000    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zw5b\" (UniqueName: \"kubernetes.io/projected/f86c296b-b555-4f4a-8d88-d24967eed2bb-kube-api-access-8zw5b\") pod \"f86c296b-b555-4f4a-8d88-d24967eed2bb\" (UID: \"f86c296b-b555-4f4a-8d88-d24967eed2bb\") "
Sep 09 21:40:02.266144 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.266092    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f86c296b-b555-4f4a-8d88-d24967eed2bb-bundle\") pod \"f86c296b-b555-4f4a-8d88-d24967eed2bb\" (UID: \"f86c296b-b555-4f4a-8d88-d24967eed2bb\") "
Sep 09 21:40:02.266289 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.266270    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0446eae-86a6-4568-b754-d4ce715e75ec-bundle\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:40:02.266289 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.266287    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/441de817-8378-4459-9103-660e9fcc735f-util\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:40:02.266457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.266295    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0446eae-86a6-4568-b754-d4ce715e75ec-util\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:40:02.266457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.266304    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hcqr\" (UniqueName: \"kubernetes.io/projected/4eba010d-604c-4915-bc95-1d51ee05faf0-kube-api-access-8hcqr\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:40:02.266457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.266313    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4eba010d-604c-4915-bc95-1d51ee05faf0-util\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:40:02.266457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.266321    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w85h8\" (UniqueName: \"kubernetes.io/projected/441de817-8378-4459-9103-660e9fcc735f-kube-api-access-w85h8\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:40:02.266457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.266329    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knkmk\" (UniqueName: \"kubernetes.io/projected/d0446eae-86a6-4568-b754-d4ce715e75ec-kube-api-access-knkmk\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:40:02.266457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.266339    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/441de817-8378-4459-9103-660e9fcc735f-bundle\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:40:02.266457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.266352    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4eba010d-604c-4915-bc95-1d51ee05faf0-bundle\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:40:02.266742 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.266557    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f86c296b-b555-4f4a-8d88-d24967eed2bb-bundle" (OuterVolumeSpecName: "bundle") pod "f86c296b-b555-4f4a-8d88-d24967eed2bb" (UID: "f86c296b-b555-4f4a-8d88-d24967eed2bb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:40:02.268162 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.268128    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f86c296b-b555-4f4a-8d88-d24967eed2bb-kube-api-access-8zw5b" (OuterVolumeSpecName: "kube-api-access-8zw5b") pod "f86c296b-b555-4f4a-8d88-d24967eed2bb" (UID: "f86c296b-b555-4f4a-8d88-d24967eed2bb"). InnerVolumeSpecName "kube-api-access-8zw5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:40:02.271751 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.271724    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f86c296b-b555-4f4a-8d88-d24967eed2bb-util" (OuterVolumeSpecName: "util") pod "f86c296b-b555-4f4a-8d88-d24967eed2bb" (UID: "f86c296b-b555-4f4a-8d88-d24967eed2bb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:40:02.366914 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.366881    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f86c296b-b555-4f4a-8d88-d24967eed2bb-bundle\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:40:02.366914 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.366909    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f86c296b-b555-4f4a-8d88-d24967eed2bb-util\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:40:02.366914 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.366923    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zw5b\" (UniqueName: \"kubernetes.io/projected/f86c296b-b555-4f4a-8d88-d24967eed2bb-kube-api-access-8zw5b\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:40:02.872755 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.872727    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq"
Sep 09 21:40:02.872931 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.872768    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amnlpq" event={"ID":"d0446eae-86a6-4568-b754-d4ce715e75ec","Type":"ContainerDied","Data":"aef4a9edae2da4fc46bd79449a6c55b96a95d4d32139283d8f56fbfe8ccbde10"}
Sep 09 21:40:02.872931 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.872809    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aef4a9edae2da4fc46bd79449a6c55b96a95d4d32139283d8f56fbfe8ccbde10"
Sep 09 21:40:02.874728 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.874708    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47"
Sep 09 21:40:02.874728 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.874715    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxpf47" event={"ID":"4eba010d-604c-4915-bc95-1d51ee05faf0","Type":"ContainerDied","Data":"5916a66ab2670cf7b042c849267cd1354d696c19dbee28c273c18a209a8366a9"}
Sep 09 21:40:02.874893 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.874753    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5916a66ab2670cf7b042c849267cd1354d696c19dbee28c273c18a209a8366a9"
Sep 09 21:40:02.876724 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.876707    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9"
Sep 09 21:40:02.876834 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.876735    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75smc9" event={"ID":"441de817-8378-4459-9103-660e9fcc735f","Type":"ContainerDied","Data":"525840d99f9787e9c53abf7183ad18e08fb3c85bd890bba48fccf2918d25afcd"}
Sep 09 21:40:02.876834 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.876769    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="525840d99f9787e9c53abf7183ad18e08fb3c85bd890bba48fccf2918d25afcd"
Sep 09 21:40:02.878801 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.878780    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp" event={"ID":"f86c296b-b555-4f4a-8d88-d24967eed2bb","Type":"ContainerDied","Data":"e93733c0d884e46dd160d3059f0747793a0847b51856e085b127cbc24c69b284"}
Sep 09 21:40:02.878894 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.878805    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6kzdtp"
Sep 09 21:40:02.878894 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:02.878817    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e93733c0d884e46dd160d3059f0747793a0847b51856e085b127cbc24c69b284"
Sep 09 21:40:17.622627 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.622589    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fl4hs"]
Sep 09 21:40:17.624082 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624042    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="441de817-8378-4459-9103-660e9fcc735f" containerName="util"
Sep 09 21:40:17.624220 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624073    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="441de817-8378-4459-9103-660e9fcc735f" containerName="util"
Sep 09 21:40:17.624266 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624229    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0446eae-86a6-4568-b754-d4ce715e75ec" containerName="extract"
Sep 09 21:40:17.624266 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624249    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0446eae-86a6-4568-b754-d4ce715e75ec" containerName="extract"
Sep 09 21:40:17.624358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624268    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f86c296b-b555-4f4a-8d88-d24967eed2bb" containerName="extract"
Sep 09 21:40:17.624358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624276    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f86c296b-b555-4f4a-8d88-d24967eed2bb" containerName="extract"
Sep 09 21:40:17.624358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624302    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f86c296b-b555-4f4a-8d88-d24967eed2bb" containerName="util"
Sep 09 21:40:17.624358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624310    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f86c296b-b555-4f4a-8d88-d24967eed2bb" containerName="util"
Sep 09 21:40:17.624358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624318    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0446eae-86a6-4568-b754-d4ce715e75ec" containerName="util"
Sep 09 21:40:17.624358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624326    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0446eae-86a6-4568-b754-d4ce715e75ec" containerName="util"
Sep 09 21:40:17.624358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624334    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eba010d-604c-4915-bc95-1d51ee05faf0" containerName="pull"
Sep 09 21:40:17.624358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624342    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eba010d-604c-4915-bc95-1d51ee05faf0" containerName="pull"
Sep 09 21:40:17.624358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624350    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="441de817-8378-4459-9103-660e9fcc735f" containerName="pull"
Sep 09 21:40:17.624358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624357    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="441de817-8378-4459-9103-660e9fcc735f" containerName="pull"
Sep 09 21:40:17.624698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624365    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eba010d-604c-4915-bc95-1d51ee05faf0" containerName="extract"
Sep 09 21:40:17.624698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624373    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eba010d-604c-4915-bc95-1d51ee05faf0" containerName="extract"
Sep 09 21:40:17.624698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624382    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="441de817-8378-4459-9103-660e9fcc735f" containerName="extract"
Sep 09 21:40:17.624698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624389    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="441de817-8378-4459-9103-660e9fcc735f" containerName="extract"
Sep 09 21:40:17.624698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624402    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eba010d-604c-4915-bc95-1d51ee05faf0" containerName="util"
Sep 09 21:40:17.624698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624412    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eba010d-604c-4915-bc95-1d51ee05faf0" containerName="util"
Sep 09 21:40:17.624698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624420    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f86c296b-b555-4f4a-8d88-d24967eed2bb" containerName="pull"
Sep 09 21:40:17.624698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624427    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f86c296b-b555-4f4a-8d88-d24967eed2bb" containerName="pull"
Sep 09 21:40:17.624698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624438    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0446eae-86a6-4568-b754-d4ce715e75ec" containerName="pull"
Sep 09 21:40:17.624698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624445    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0446eae-86a6-4568-b754-d4ce715e75ec" containerName="pull"
Sep 09 21:40:17.624698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624530    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0446eae-86a6-4568-b754-d4ce715e75ec" containerName="extract"
Sep 09 21:40:17.624698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624545    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="441de817-8378-4459-9103-660e9fcc735f" containerName="extract"
Sep 09 21:40:17.624698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624553    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eba010d-604c-4915-bc95-1d51ee05faf0" containerName="extract"
Sep 09 21:40:17.624698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.624562    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f86c296b-b555-4f4a-8d88-d24967eed2bb" containerName="extract"
Sep 09 21:40:17.629218 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.629200    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fl4hs"
Sep 09 21:40:17.631820 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.631791    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 09 21:40:17.631946 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.631792    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 09 21:40:17.632019 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.631953    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 09 21:40:17.632093 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.632078    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5kkdn\""
Sep 09 21:40:17.632640 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.632623    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fl4hs"]
Sep 09 21:40:17.798151 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.798119    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv2hg\" (UniqueName: \"kubernetes.io/projected/2b9d853d-c684-4128-8bbd-5ab7f3fdbc8f-kube-api-access-zv2hg\") pod \"dns-operator-controller-manager-7b84cd7c9c-fl4hs\" (UID: \"2b9d853d-c684-4128-8bbd-5ab7f3fdbc8f\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fl4hs"
Sep 09 21:40:17.899148 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.899079    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv2hg\" (UniqueName: \"kubernetes.io/projected/2b9d853d-c684-4128-8bbd-5ab7f3fdbc8f-kube-api-access-zv2hg\") pod \"dns-operator-controller-manager-7b84cd7c9c-fl4hs\" (UID: \"2b9d853d-c684-4128-8bbd-5ab7f3fdbc8f\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fl4hs"
Sep 09 21:40:17.914519 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.914490    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv2hg\" (UniqueName: \"kubernetes.io/projected/2b9d853d-c684-4128-8bbd-5ab7f3fdbc8f-kube-api-access-zv2hg\") pod \"dns-operator-controller-manager-7b84cd7c9c-fl4hs\" (UID: \"2b9d853d-c684-4128-8bbd-5ab7f3fdbc8f\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fl4hs"
Sep 09 21:40:17.940126 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:17.940096    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fl4hs"
Sep 09 21:40:18.217322 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:18.217300    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fl4hs"]
Sep 09 21:40:18.219631 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:40:18.219606    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b9d853d_c684_4128_8bbd_5ab7f3fdbc8f.slice/crio-ec9e3839ff483b2863a191716806ef0add604040fa7d978a3438c543e72b123c WatchSource:0}: Error finding container ec9e3839ff483b2863a191716806ef0add604040fa7d978a3438c543e72b123c: Status 404 returned error can't find the container with id ec9e3839ff483b2863a191716806ef0add604040fa7d978a3438c543e72b123c
Sep 09 21:40:18.967891 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:18.967856    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fl4hs" event={"ID":"2b9d853d-c684-4128-8bbd-5ab7f3fdbc8f","Type":"ContainerStarted","Data":"ec9e3839ff483b2863a191716806ef0add604040fa7d978a3438c543e72b123c"}
Sep 09 21:40:19.557221 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.557192    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47"]
Sep 09 21:40:19.560857 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.560835    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47"
Sep 09 21:40:19.563407 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.563342    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 09 21:40:19.563407 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.563362    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 09 21:40:19.563557 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.563343    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4r4xn\""
Sep 09 21:40:19.567684 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.567643    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47"]
Sep 09 21:40:19.714240 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.714210    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/074ab1c6-f610-4bc1-9cca-56eefee785e8-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-j7w47\" (UID: \"074ab1c6-f610-4bc1-9cca-56eefee785e8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47"
Sep 09 21:40:19.714240 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.714246    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57lbk\" (UniqueName: \"kubernetes.io/projected/074ab1c6-f610-4bc1-9cca-56eefee785e8-kube-api-access-57lbk\") pod \"kuadrant-console-plugin-fbdd4f445-j7w47\" (UID: \"074ab1c6-f610-4bc1-9cca-56eefee785e8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47"
Sep 09 21:40:19.714458 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.714386    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/074ab1c6-f610-4bc1-9cca-56eefee785e8-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-j7w47\" (UID: \"074ab1c6-f610-4bc1-9cca-56eefee785e8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47"
Sep 09 21:40:19.815704 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.815616    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/074ab1c6-f610-4bc1-9cca-56eefee785e8-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-j7w47\" (UID: \"074ab1c6-f610-4bc1-9cca-56eefee785e8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47"
Sep 09 21:40:19.815848 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.815786    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/074ab1c6-f610-4bc1-9cca-56eefee785e8-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-j7w47\" (UID: \"074ab1c6-f610-4bc1-9cca-56eefee785e8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47"
Sep 09 21:40:19.815848 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.815810    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57lbk\" (UniqueName: \"kubernetes.io/projected/074ab1c6-f610-4bc1-9cca-56eefee785e8-kube-api-access-57lbk\") pod \"kuadrant-console-plugin-fbdd4f445-j7w47\" (UID: \"074ab1c6-f610-4bc1-9cca-56eefee785e8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47"
Sep 09 21:40:19.816329 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.816300    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/074ab1c6-f610-4bc1-9cca-56eefee785e8-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-j7w47\" (UID: \"074ab1c6-f610-4bc1-9cca-56eefee785e8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47"
Sep 09 21:40:19.818443 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.818405    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/074ab1c6-f610-4bc1-9cca-56eefee785e8-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-j7w47\" (UID: \"074ab1c6-f610-4bc1-9cca-56eefee785e8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47"
Sep 09 21:40:19.823359 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.823334    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57lbk\" (UniqueName: \"kubernetes.io/projected/074ab1c6-f610-4bc1-9cca-56eefee785e8-kube-api-access-57lbk\") pod \"kuadrant-console-plugin-fbdd4f445-j7w47\" (UID: \"074ab1c6-f610-4bc1-9cca-56eefee785e8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47"
Sep 09 21:40:19.873308 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:19.873285    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47"
Sep 09 21:40:20.926693 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:20.926602    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47"]
Sep 09 21:40:20.929655 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:40:20.929627    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod074ab1c6_f610_4bc1_9cca_56eefee785e8.slice/crio-09d3bd19f64c9fca42608538fa6a7195772dd6011c5b25e7800789ae88acdbc6 WatchSource:0}: Error finding container 09d3bd19f64c9fca42608538fa6a7195772dd6011c5b25e7800789ae88acdbc6: Status 404 returned error can't find the container with id 09d3bd19f64c9fca42608538fa6a7195772dd6011c5b25e7800789ae88acdbc6
Sep 09 21:40:20.982025 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:20.981963    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fl4hs" event={"ID":"2b9d853d-c684-4128-8bbd-5ab7f3fdbc8f","Type":"ContainerStarted","Data":"19bb31d1d9304b2c32117e078c85852e9d545fc580934293eb95a167f13dda9f"}
Sep 09 21:40:20.982242 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:20.982065    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fl4hs"
Sep 09 21:40:20.983418 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:20.983385    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47" event={"ID":"074ab1c6-f610-4bc1-9cca-56eefee785e8","Type":"ContainerStarted","Data":"09d3bd19f64c9fca42608538fa6a7195772dd6011c5b25e7800789ae88acdbc6"}
Sep 09 21:40:21.000287 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:21.000235    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fl4hs" podStartSLOduration=1.491275865 podStartE2EDuration="4.000216033s" totalImagesPullingTime="2.508940168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:40:20.998310154 +0000 UTC m=+440.471275065" watchObservedRunningTime="2026-09-09 21:40:21.000216033 +0000 UTC m=+440.473180947"
Sep 09 21:40:26.014995 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:26.014939    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47" event={"ID":"074ab1c6-f610-4bc1-9cca-56eefee785e8","Type":"ContainerStarted","Data":"cd3765ca390d0f71c8d60d308439414ca70ab74cf710edee8e7f0844fed3b15f"}
Sep 09 21:40:26.030657 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:26.030596    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j7w47" podStartSLOduration=2.083916785 podStartE2EDuration="7.030577333s" totalImagesPullingTime="4.946660548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:40:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:40:26.029997722 +0000 UTC m=+445.502962639" watchObservedRunningTime="2026-09-09 21:40:26.030577333 +0000 UTC m=+445.503542245"
Sep 09 21:40:31.993880 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:31.993846    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fl4hs"
Sep 09 21:40:59.327947 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.327914    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"]
Sep 09 21:40:59.333815 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.333793    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.336589 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.336568    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tjpfk\""
Sep 09 21:40:59.343096 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.343068    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"]
Sep 09 21:40:59.458760 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.458722    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.458953 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.458765    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.458953 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.458825    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.458953 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.458854    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgb5z\" (UniqueName: \"kubernetes.io/projected/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-kube-api-access-cgb5z\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.458953 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.458936    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.459127 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.458987    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.459127 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.459013    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.459127 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.459038    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.459127 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.459061    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.559900 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.559867    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.560127 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.559913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.560127 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.559938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgb5z\" (UniqueName: \"kubernetes.io/projected/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-kube-api-access-cgb5z\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.560127 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.560111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.560305 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.560137    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.560305 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.560162    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.560305 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.560205    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.560305 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.560238    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.560516 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.560309    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.560573 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.560517    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.560636 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.560585    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.560713 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.560689    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.560774 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.560702    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.560774 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.560757    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.562443 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.562413    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.562566 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.562549    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.573853 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.573825    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgb5z\" (UniqueName: \"kubernetes.io/projected/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-kube-api-access-cgb5z\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.575663 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.575641    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-vxvgt\" (UID: \"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.644728 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.644651    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:40:59.918001 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.917958    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"]
Sep 09 21:40:59.920437 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:40:59.920409    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3be0db2_ed98_4a15_ad0f_7f1b3e3833ab.slice/crio-fa5a6a14f968c1a73a25d2aa2c602c6bf808eb9870a530b9fbd23cd98267e2c1 WatchSource:0}: Error finding container fa5a6a14f968c1a73a25d2aa2c602c6bf808eb9870a530b9fbd23cd98267e2c1: Status 404 returned error can't find the container with id fa5a6a14f968c1a73a25d2aa2c602c6bf808eb9870a530b9fbd23cd98267e2c1
Sep 09 21:40:59.922492 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.922460    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:40:59.922572 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.922523    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:40:59.922572 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:40:59.922550    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:41:00.189645 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:00.189561    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt" event={"ID":"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab","Type":"ContainerStarted","Data":"97dc583988d58f41fb9b61f48a880a656de46d38397c912c62d24dfc7083882b"}
Sep 09 21:41:00.189645 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:00.189611    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt" event={"ID":"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab","Type":"ContainerStarted","Data":"fa5a6a14f968c1a73a25d2aa2c602c6bf808eb9870a530b9fbd23cd98267e2c1"}
Sep 09 21:41:00.208230 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:00.208182    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt" podStartSLOduration=1.20816743 podStartE2EDuration="1.20816743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:40:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:41:00.20532295 +0000 UTC m=+479.678287895" watchObservedRunningTime="2026-09-09 21:41:00.20816743 +0000 UTC m=+479.681132340"
Sep 09 21:41:00.645208 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:00.645175    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:41:00.649917 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:00.649893    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:41:01.195027 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:01.194994    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:41:01.196033 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:01.196012    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:41:01.527335 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:01.527282    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:41:01.529896 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:01.529876    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"
Sep 09 21:41:01.531849 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:01.531829    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 09 21:41:01.538009 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:01.537988    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:41:01.565094 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:01.565069    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:41:01.678134 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:01.678103    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htc4p\" (UniqueName: \"kubernetes.io/projected/e7157711-a94f-4a96-a2e5-3875233ebf2c-kube-api-access-htc4p\") pod \"limitador-limitador-84994f7ffc-lkzhw\" (UID: \"e7157711-a94f-4a96-a2e5-3875233ebf2c\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"
Sep 09 21:41:01.678485 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:01.678143    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7157711-a94f-4a96-a2e5-3875233ebf2c-config-file\") pod \"limitador-limitador-84994f7ffc-lkzhw\" (UID: \"e7157711-a94f-4a96-a2e5-3875233ebf2c\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"
Sep 09 21:41:01.779081 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:01.779011    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htc4p\" (UniqueName: \"kubernetes.io/projected/e7157711-a94f-4a96-a2e5-3875233ebf2c-kube-api-access-htc4p\") pod \"limitador-limitador-84994f7ffc-lkzhw\" (UID: \"e7157711-a94f-4a96-a2e5-3875233ebf2c\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"
Sep 09 21:41:01.779081 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:01.779056    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7157711-a94f-4a96-a2e5-3875233ebf2c-config-file\") pod \"limitador-limitador-84994f7ffc-lkzhw\" (UID: \"e7157711-a94f-4a96-a2e5-3875233ebf2c\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"
Sep 09 21:41:01.779740 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:01.779721    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7157711-a94f-4a96-a2e5-3875233ebf2c-config-file\") pod \"limitador-limitador-84994f7ffc-lkzhw\" (UID: \"e7157711-a94f-4a96-a2e5-3875233ebf2c\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"
Sep 09 21:41:01.787306 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:01.787275    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htc4p\" (UniqueName: \"kubernetes.io/projected/e7157711-a94f-4a96-a2e5-3875233ebf2c-kube-api-access-htc4p\") pod \"limitador-limitador-84994f7ffc-lkzhw\" (UID: \"e7157711-a94f-4a96-a2e5-3875233ebf2c\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"
Sep 09 21:41:01.840007 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:01.839962    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"
Sep 09 21:41:02.112084 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:02.112055    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:41:02.113104 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:41:02.113081    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7157711_a94f_4a96_a2e5_3875233ebf2c.slice/crio-dd258d2f850cc89983ad629a2d863cddb221502e847afebb35ed00092149ae2e WatchSource:0}: Error finding container dd258d2f850cc89983ad629a2d863cddb221502e847afebb35ed00092149ae2e: Status 404 returned error can't find the container with id dd258d2f850cc89983ad629a2d863cddb221502e847afebb35ed00092149ae2e
Sep 09 21:41:02.201292 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:02.201256    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-lkzhw" event={"ID":"e7157711-a94f-4a96-a2e5-3875233ebf2c","Type":"ContainerStarted","Data":"dd258d2f850cc89983ad629a2d863cddb221502e847afebb35ed00092149ae2e"}
Sep 09 21:41:04.214377 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:04.214335    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-lkzhw" event={"ID":"e7157711-a94f-4a96-a2e5-3875233ebf2c","Type":"ContainerStarted","Data":"83bff378e6c63b7963926aa134a7c52676e9a60f09dca2cc260ca3e10e0fa5f5"}
Sep 09 21:41:04.214775 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:04.214410    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"
Sep 09 21:41:04.228623 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:04.228431    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-lkzhw" podStartSLOduration=2.000867294 podStartE2EDuration="3.228410788s" totalImagesPullingTime="1.227543494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:41:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:41:04.228306738 +0000 UTC m=+483.701271649" watchObservedRunningTime="2026-09-09 21:41:04.228410788 +0000 UTC m=+483.701375700"
Sep 09 21:41:15.221567 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:41:15.221534    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"
Sep 09 21:42:12.277697 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:12.277666    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:42:21.627828 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:21.627734    2578 generic.go:356] "Generic (PLEG): container finished" podID="e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab" containerID="97dc583988d58f41fb9b61f48a880a656de46d38397c912c62d24dfc7083882b" exitCode=0
Sep 09 21:42:21.627828 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:21.627792    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt" event={"ID":"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab","Type":"ContainerDied","Data":"97dc583988d58f41fb9b61f48a880a656de46d38397c912c62d24dfc7083882b"}
Sep 09 21:42:21.628283 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:21.628167    2578 scope.go:117] "RemoveContainer" containerID="97dc583988d58f41fb9b61f48a880a656de46d38397c912c62d24dfc7083882b"
Sep 09 21:42:21.635106 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:21.629637    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:42:21.635106 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:21.629807    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:42:21.635106 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:21.629900    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:42:22.639999 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:22.639255    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt" event={"ID":"e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab","Type":"ContainerStarted","Data":"67de31e5973f87912c8ddbd40e562514d85951f9b64522269dcc56a33f58dd15"}
Sep 09 21:42:22.645260 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:22.645226    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:42:22.645442 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:22.645360    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-vxvgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 09 21:42:22.645442 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:22.645404    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt" podUID="e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 09 21:42:23.646252 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:23.646211    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-vxvgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 09 21:42:23.646716 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:23.646289    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt" podUID="e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 09 21:42:24.645516 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:24.645471    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-vxvgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 09 21:42:24.646211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:24.645790    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt" podUID="e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 09 21:42:25.645376 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:25.645336    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-vxvgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 09 21:42:25.646074 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:25.646031    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt" podUID="e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 09 21:42:26.645610 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:26.645573    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-vxvgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 09 21:42:26.646200 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:26.646169    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt" podUID="e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 09 21:42:27.645235 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:27.645195    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-vxvgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Sep 09 21:42:27.645435 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:27.645259    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt" podUID="e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Sep 09 21:42:28.648908 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:28.648872    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:42:28.649280 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:28.649152    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:42:28.649775 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:28.649753    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-vxvgt"
Sep 09 21:42:32.067716 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:32.067685    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:42:52.882050 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:52.882021    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"]
Sep 09 21:42:52.885884 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:52.885865    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:52.889399 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:52.889187    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wqwhc\""
Sep 09 21:42:52.889399 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:52.889202    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 09 21:42:52.889399 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:52.889264    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 09 21:42:52.889399 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:52.889283    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 09 21:42:52.897136 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:52.897117    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"]
Sep 09 21:42:53.003931 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.003906    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8615776e-9ce8-4608-90c4-8fa84c8a6341-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.004089 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.003943    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv6s5\" (UniqueName: \"kubernetes.io/projected/8615776e-9ce8-4608-90c4-8fa84c8a6341-kube-api-access-pv6s5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.004089 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.003990    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8615776e-9ce8-4608-90c4-8fa84c8a6341-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.004089 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.004051    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8615776e-9ce8-4608-90c4-8fa84c8a6341-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.004257 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.004114    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8615776e-9ce8-4608-90c4-8fa84c8a6341-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.004257 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.004214    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8615776e-9ce8-4608-90c4-8fa84c8a6341-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.104937 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.104898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8615776e-9ce8-4608-90c4-8fa84c8a6341-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.105139 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.104949    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv6s5\" (UniqueName: \"kubernetes.io/projected/8615776e-9ce8-4608-90c4-8fa84c8a6341-kube-api-access-pv6s5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.105139 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.105001    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8615776e-9ce8-4608-90c4-8fa84c8a6341-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.105139 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.105030    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8615776e-9ce8-4608-90c4-8fa84c8a6341-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.105139 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.105082    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8615776e-9ce8-4608-90c4-8fa84c8a6341-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.105423 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.105166    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8615776e-9ce8-4608-90c4-8fa84c8a6341-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.105492 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.105454    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8615776e-9ce8-4608-90c4-8fa84c8a6341-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.105552 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.105513    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8615776e-9ce8-4608-90c4-8fa84c8a6341-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.105595 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.105557    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8615776e-9ce8-4608-90c4-8fa84c8a6341-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.107308 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.107281    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8615776e-9ce8-4608-90c4-8fa84c8a6341-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.107443 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.107426    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8615776e-9ce8-4608-90c4-8fa84c8a6341-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.112049 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.112026    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv6s5\" (UniqueName: \"kubernetes.io/projected/8615776e-9ce8-4608-90c4-8fa84c8a6341-kube-api-access-pv6s5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ldnp8\" (UID: \"8615776e-9ce8-4608-90c4-8fa84c8a6341\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.198583 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.198511    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:42:53.473632 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.471604    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"]
Sep 09 21:42:53.474424 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:42:53.474394    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8615776e_9ce8_4608_90c4_8fa84c8a6341.slice/crio-3d97d88d93c395540825730896a238b8c369c7ed1652b148a5473fe5331fa3af WatchSource:0}: Error finding container 3d97d88d93c395540825730896a238b8c369c7ed1652b148a5473fe5331fa3af: Status 404 returned error can't find the container with id 3d97d88d93c395540825730896a238b8c369c7ed1652b148a5473fe5331fa3af
Sep 09 21:42:53.666708 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.666673    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:42:53.816254 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:53.816217    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8" event={"ID":"8615776e-9ce8-4608-90c4-8fa84c8a6341","Type":"ContainerStarted","Data":"3d97d88d93c395540825730896a238b8c369c7ed1652b148a5473fe5331fa3af"}
Sep 09 21:42:54.823141 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:54.823100    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8" event={"ID":"8615776e-9ce8-4608-90c4-8fa84c8a6341","Type":"ContainerStarted","Data":"76c7173d4f297769724b6b0f0310f1585efe238e3dea6cf01cbf2790cbb4289a"}
Sep 09 21:42:54.842218 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:54.842169    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8" podStartSLOduration=1.81505921 podStartE2EDuration="2.842154075s" totalImagesPullingTime="1.027094865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:42:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:42:54.839246735 +0000 UTC m=+594.312211639" watchObservedRunningTime="2026-09-09 21:42:54.842154075 +0000 UTC m=+594.315118984"
Sep 09 21:42:57.395812 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:42:57.395782    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:43:00.964358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:43:00.964327    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 21:43:00.967546 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:43:00.967521    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 21:43:03.199290 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:43:03.199255    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:43:03.199683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:43:03.199313    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:43:03.212175 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:43:03.212151    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:43:03.884731 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:43:03.884699    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ldnp8"
Sep 09 21:43:19.168199 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:43:19.168163    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:43:31.674431 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:43:31.674348    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:43:47.369901 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:43:47.369867    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:44:08.174470 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:44:08.174435    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:45:00.144629 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:00.144546    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816505-l9dxb"]
Sep 09 21:45:00.148041 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:00.148017    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816505-l9dxb"
Sep 09 21:45:00.150316 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:00.150293    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65jqq\""
Sep 09 21:45:00.153937 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:00.153911    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816505-l9dxb"]
Sep 09 21:45:00.160241 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:00.160183    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816505-l9dxb"
Sep 09 21:45:00.432842 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:00.432810    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816505-l9dxb"]
Sep 09 21:45:00.435092 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:45:00.435059    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20cb7fae_d483_4337_b7fc_dffb21361761.slice/crio-fb0d407b43f8204a02934c7e6c504a88e9925cc00a9c9a9e9c31c0cf5ab2bc8d WatchSource:0}: Error finding container fb0d407b43f8204a02934c7e6c504a88e9925cc00a9c9a9e9c31c0cf5ab2bc8d: Status 404 returned error can't find the container with id fb0d407b43f8204a02934c7e6c504a88e9925cc00a9c9a9e9c31c0cf5ab2bc8d
Sep 09 21:45:00.437107 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:00.437087    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 21:45:00.514274 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:00.514238    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816505-l9dxb" event={"ID":"20cb7fae-d483-4337-b7fc-dffb21361761","Type":"ContainerStarted","Data":"fb0d407b43f8204a02934c7e6c504a88e9925cc00a9c9a9e9c31c0cf5ab2bc8d"}
Sep 09 21:45:03.532748 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:03.532700    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816505-l9dxb" event={"ID":"20cb7fae-d483-4337-b7fc-dffb21361761","Type":"ContainerStarted","Data":"ec786453fcde240587937e02cd0518dd189d9d32edddfc9fdb8dfd1b4cda2a2c"}
Sep 09 21:45:03.545943 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:03.545892    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816505-l9dxb" podStartSLOduration=0.991790525 podStartE2EDuration="3.545874659s" totalImagesPullingTime="2.554084134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:45:03.545518897 +0000 UTC m=+723.018483808" watchObservedRunningTime="2026-09-09 21:45:03.545874659 +0000 UTC m=+723.018839574"
Sep 09 21:45:04.539362 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:04.539326    2578 generic.go:356] "Generic (PLEG): container finished" podID="20cb7fae-d483-4337-b7fc-dffb21361761" containerID="ec786453fcde240587937e02cd0518dd189d9d32edddfc9fdb8dfd1b4cda2a2c" exitCode=0
Sep 09 21:45:04.539743 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:04.539374    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816505-l9dxb" event={"ID":"20cb7fae-d483-4337-b7fc-dffb21361761","Type":"ContainerDied","Data":"ec786453fcde240587937e02cd0518dd189d9d32edddfc9fdb8dfd1b4cda2a2c"}
Sep 09 21:45:05.678602 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:05.678579    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816505-l9dxb"
Sep 09 21:45:06.551195 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:06.551167    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816505-l9dxb"
Sep 09 21:45:06.551366 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:06.551235    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816505-l9dxb" event={"ID":"20cb7fae-d483-4337-b7fc-dffb21361761","Type":"ContainerDied","Data":"fb0d407b43f8204a02934c7e6c504a88e9925cc00a9c9a9e9c31c0cf5ab2bc8d"}
Sep 09 21:45:06.551366 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:06.551274    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb0d407b43f8204a02934c7e6c504a88e9925cc00a9c9a9e9c31c0cf5ab2bc8d"
Sep 09 21:45:46.663365 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.663323    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"]
Sep 09 21:45:46.663828 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.663699    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20cb7fae-d483-4337-b7fc-dffb21361761" containerName="cleanup"
Sep 09 21:45:46.663828 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.663711    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="20cb7fae-d483-4337-b7fc-dffb21361761" containerName="cleanup"
Sep 09 21:45:46.663828 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.663784    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="20cb7fae-d483-4337-b7fc-dffb21361761" containerName="cleanup"
Sep 09 21:45:46.668220 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.668200    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.672267 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.672245    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-3fa986-openshift-default-dockercfg-2d7lb\""
Sep 09 21:45:46.693019 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.692994    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"]
Sep 09 21:45:46.800981 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.800944    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"]
Sep 09 21:45:46.804037 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.804005    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:46.808495 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.807954    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-390abc-openshift-default-dockercfg-4nshd\""
Sep 09 21:45:46.812229 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.812206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-credential-socket\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.812353 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.812239    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-workload-certs\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.812411 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.812341    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-workload-socket\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.812474 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.812416    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-envoy\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.812474 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.812439    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hgtj\" (UniqueName: \"kubernetes.io/projected/787bc763-f469-4eb4-a3cb-bef97d299dd0-kube-api-access-5hgtj\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.812603 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.812475    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-data\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.812603 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.812537    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-podinfo\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.812603 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.812572    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-token\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.812603 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.812601    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/787bc763-f469-4eb4-a3cb-bef97d299dd0-istiod-ca-cert\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.823423 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.823398    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"]
Sep 09 21:45:46.880365 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.880328    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"]
Sep 09 21:45:46.882866 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.882847    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:46.885582 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.885554    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-2aaab1-openshift-default-dockercfg-jbzlt\""
Sep 09 21:45:46.896092 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.896072    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"]
Sep 09 21:45:46.914066 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.913998    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-workload-socket\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.914066 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914032    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06282b23-1694-4fc2-b3c3-c17e924cca00-istiod-ca-cert\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:46.914214 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914078    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-workload-certs\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:46.914214 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914129    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-envoy\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.914214 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914154    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hgtj\" (UniqueName: \"kubernetes.io/projected/787bc763-f469-4eb4-a3cb-bef97d299dd0-kube-api-access-5hgtj\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.914214 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914179    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-envoy\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:46.914214 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914202    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-token\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:46.914463 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914232    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-data\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.914463 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914321    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-podinfo\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.914463 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914370    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-token\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.914463 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-credential-socket\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:46.914463 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914413    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-workload-socket\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.914463 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914438    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-podinfo\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:46.914766 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914466    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/787bc763-f469-4eb4-a3cb-bef97d299dd0-istiod-ca-cert\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.914766 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914505    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-workload-socket\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:46.914766 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914530    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-data\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:46.914766 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914573    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-credential-socket\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.914766 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-workload-certs\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.914766 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914649    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97hrc\" (UniqueName: \"kubernetes.io/projected/06282b23-1694-4fc2-b3c3-c17e924cca00-kube-api-access-97hrc\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:46.914766 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914702    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-data\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.915145 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.914894    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-credential-socket\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.915145 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.915106    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-workload-certs\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.915479 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.915456    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/787bc763-f469-4eb4-a3cb-bef97d299dd0-istiod-ca-cert\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.916459 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.916439    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-envoy\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.916903 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.916888    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-podinfo\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.921609 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.921589    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hgtj\" (UniqueName: \"kubernetes.io/projected/787bc763-f469-4eb4-a3cb-bef97d299dd0-kube-api-access-5hgtj\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.924356 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.924322    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-token\") pod \"e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") " pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:46.978511 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:46.978480    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:47.015348 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015318    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-token\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.015535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-workload-certs\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.015535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015388    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-envoy\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.015535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015403    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-token\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.015535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015441    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvl6s\" (UniqueName: \"kubernetes.io/projected/40c04813-7e10-4669-a048-93ad3f50c5b4-kube-api-access-fvl6s\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.015535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015467    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-workload-socket\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.015535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015493    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-envoy\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.015535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015526    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-credential-socket\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.015914 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015559    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c04813-7e10-4669-a048-93ad3f50c5b4-istiod-ca-cert\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.015914 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015580    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-podinfo\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.015914 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015620    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-workload-socket\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.015914 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015635    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-data\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.015914 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015680    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-workload-certs\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.015914 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015712    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-credential-socket\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.015914 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015742    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97hrc\" (UniqueName: \"kubernetes.io/projected/06282b23-1694-4fc2-b3c3-c17e924cca00-kube-api-access-97hrc\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.015914 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015754    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-workload-certs\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.015914 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015779    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06282b23-1694-4fc2-b3c3-c17e924cca00-istiod-ca-cert\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.015914 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015829    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-podinfo\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.015914 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015849    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-credential-socket\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.015914 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.015866    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-data\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.016480 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.016197    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-workload-socket\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.016480 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.016277    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-data\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.016638 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.016616    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06282b23-1694-4fc2-b3c3-c17e924cca00-istiod-ca-cert\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.018106 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.018084    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-envoy\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.018420 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.018396    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-podinfo\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.025032 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.024985    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97hrc\" (UniqueName: \"kubernetes.io/projected/06282b23-1694-4fc2-b3c3-c17e924cca00-kube-api-access-97hrc\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.025172 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.025153    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-token\") pod \"e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") " pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.116422 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.116392    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:47.116600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.116576    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-workload-socket\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.116662 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.116617    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-envoy\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.116662 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.116640    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c04813-7e10-4669-a048-93ad3f50c5b4-istiod-ca-cert\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.116760 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.116696    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-workload-certs\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.116760 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.116724    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-credential-socket\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.116879 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.116767    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-podinfo\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.116879 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.116810    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-data\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.116879 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.116841    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-token\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.117049 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.116901    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvl6s\" (UniqueName: \"kubernetes.io/projected/40c04813-7e10-4669-a048-93ad3f50c5b4-kube-api-access-fvl6s\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.117293 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.117066    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-workload-socket\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.117293 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.117164    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-workload-certs\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.117293 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.117257    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-credential-socket\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.117468 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.117368    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-data\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.117468 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.117438    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c04813-7e10-4669-a048-93ad3f50c5b4-istiod-ca-cert\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.119629 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.119604    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-envoy\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.119835 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.119819    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-podinfo\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.126002 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.125959    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-token\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.126156 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.126138    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvl6s\" (UniqueName: \"kubernetes.io/projected/40c04813-7e10-4669-a048-93ad3f50c5b4-kube-api-access-fvl6s\") pod \"e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") " pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.193235 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.193206    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:47.265392 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.265179    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"]
Sep 09 21:45:47.266460 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:45:47.266401    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod787bc763_f469_4eb4_a3cb_bef97d299dd0.slice/crio-c2be9394283f821a9f660abb40d3201df52287cacf6156b22fd33b90ff55d3f8 WatchSource:0}: Error finding container c2be9394283f821a9f660abb40d3201df52287cacf6156b22fd33b90ff55d3f8: Status 404 returned error can't find the container with id c2be9394283f821a9f660abb40d3201df52287cacf6156b22fd33b90ff55d3f8
Sep 09 21:45:47.269095 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.269063    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:45:47.269153 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.269127    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:45:47.269189 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.269161    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:45:47.396576 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.396553    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"]
Sep 09 21:45:47.398771 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:45:47.398741    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06282b23_1694_4fc2_b3c3_c17e924cca00.slice/crio-7ef5e13e3cd15a51fb1548791bb49dccbd2bb4508f06796c9bd096958b665a8e WatchSource:0}: Error finding container 7ef5e13e3cd15a51fb1548791bb49dccbd2bb4508f06796c9bd096958b665a8e: Status 404 returned error can't find the container with id 7ef5e13e3cd15a51fb1548791bb49dccbd2bb4508f06796c9bd096958b665a8e
Sep 09 21:45:47.401609 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.401576    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:45:47.401718 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.401640    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:45:47.401718 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.401669    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:45:47.476596 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.476570    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"]
Sep 09 21:45:47.477585 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:45:47.477554    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c04813_7e10_4669_a048_93ad3f50c5b4.slice/crio-879fcf5b67fb909a3188c333c21ade952b649cb6077ef1b68a4dbcfe21eef3bb WatchSource:0}: Error finding container 879fcf5b67fb909a3188c333c21ade952b649cb6077ef1b68a4dbcfe21eef3bb: Status 404 returned error can't find the container with id 879fcf5b67fb909a3188c333c21ade952b649cb6077ef1b68a4dbcfe21eef3bb
Sep 09 21:45:47.480219 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.480186    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:45:47.480317 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.480260    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:45:47.480317 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.480304    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:45:47.772777 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.772712    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" event={"ID":"40c04813-7e10-4669-a048-93ad3f50c5b4","Type":"ContainerStarted","Data":"00b8c89d9eea7e7fe3309f7de78726d9a8cfcc95eb1066167ef353f02f740615"}
Sep 09 21:45:47.773289 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.772789    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" event={"ID":"40c04813-7e10-4669-a048-93ad3f50c5b4","Type":"ContainerStarted","Data":"879fcf5b67fb909a3188c333c21ade952b649cb6077ef1b68a4dbcfe21eef3bb"}
Sep 09 21:45:47.774817 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.774786    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" event={"ID":"787bc763-f469-4eb4-a3cb-bef97d299dd0","Type":"ContainerStarted","Data":"bebe50c6962b4d169194addd2d75de38b300d8d222a6533f2d7c114473113658"}
Sep 09 21:45:47.774954 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.774843    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" event={"ID":"787bc763-f469-4eb4-a3cb-bef97d299dd0","Type":"ContainerStarted","Data":"c2be9394283f821a9f660abb40d3201df52287cacf6156b22fd33b90ff55d3f8"}
Sep 09 21:45:47.776765 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.776729    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb" event={"ID":"06282b23-1694-4fc2-b3c3-c17e924cca00","Type":"ContainerStarted","Data":"af1f598aba0235e89dd793bb0ff0bca71e507ca350c7804a4a7d464bc1a2f955"}
Sep 09 21:45:47.776890 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.776781    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb" event={"ID":"06282b23-1694-4fc2-b3c3-c17e924cca00","Type":"ContainerStarted","Data":"7ef5e13e3cd15a51fb1548791bb49dccbd2bb4508f06796c9bd096958b665a8e"}
Sep 09 21:45:47.823159 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.823101    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" podStartSLOduration=1.8230810929999999 podStartE2EDuration="1.823081093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:45:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:45:47.796454888 +0000 UTC m=+767.269419799" watchObservedRunningTime="2026-09-09 21:45:47.823081093 +0000 UTC m=+767.296046005"
Sep 09 21:45:47.823639 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.823501    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" podStartSLOduration=1.823485861 podStartE2EDuration="1.823485861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:45:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:45:47.822728795 +0000 UTC m=+767.295693731" watchObservedRunningTime="2026-09-09 21:45:47.823485861 +0000 UTC m=+767.296450772"
Sep 09 21:45:47.842828 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.842788    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb" podStartSLOduration=1.842774331 podStartE2EDuration="1.842774331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:45:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:45:47.841808626 +0000 UTC m=+767.314773526" watchObservedRunningTime="2026-09-09 21:45:47.842774331 +0000 UTC m=+767.315739243"
Sep 09 21:45:47.978631 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:47.978597    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:48.117005 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.116892    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:48.121746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.121726    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:48.194126 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.194097    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:48.199365 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.199342    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:48.216805 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.216780    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7504-f47c63"]
Sep 09 21:45:48.228160 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.228130    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7504-f47c63"]
Sep 09 21:45:48.228277 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.228228    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7504-f47c63"
Sep 09 21:45:48.230632 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.230612    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-k4l58\""
Sep 09 21:45:48.329516 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.329483    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8th5\" (UniqueName: \"kubernetes.io/projected/7587d6d7-3f38-42d5-98c1-20b6802b09fa-kube-api-access-x8th5\") pod \"test-curl-7504-f47c63\" (UID: \"7587d6d7-3f38-42d5-98c1-20b6802b09fa\") " pod="opendatahub/test-curl-7504-f47c63"
Sep 09 21:45:48.430128 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.430051    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8th5\" (UniqueName: \"kubernetes.io/projected/7587d6d7-3f38-42d5-98c1-20b6802b09fa-kube-api-access-x8th5\") pod \"test-curl-7504-f47c63\" (UID: \"7587d6d7-3f38-42d5-98c1-20b6802b09fa\") " pod="opendatahub/test-curl-7504-f47c63"
Sep 09 21:45:48.442417 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.442393    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8th5\" (UniqueName: \"kubernetes.io/projected/7587d6d7-3f38-42d5-98c1-20b6802b09fa-kube-api-access-x8th5\") pod \"test-curl-7504-f47c63\" (UID: \"7587d6d7-3f38-42d5-98c1-20b6802b09fa\") " pod="opendatahub/test-curl-7504-f47c63"
Sep 09 21:45:48.539437 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.539394    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7504-f47c63"
Sep 09 21:45:48.782157 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.782131    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:48.782613 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.782189    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:48.783396 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.783360    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:45:48.783504 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.783424    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:45:48.808754 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.808729    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7504-f47c63"]
Sep 09 21:45:48.809807 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:45:48.809781    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7587d6d7_3f38_42d5_98c1_20b6802b09fa.slice/crio-740a1b5117e3400735df07c2c16b16171cb5246ca18a49a040914a82bac78ccb WatchSource:0}: Error finding container 740a1b5117e3400735df07c2c16b16171cb5246ca18a49a040914a82bac78ccb: Status 404 returned error can't find the container with id 740a1b5117e3400735df07c2c16b16171cb5246ca18a49a040914a82bac78ccb
Sep 09 21:45:48.983690 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:48.983663    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:49.789188 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:49.789139    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7504-f47c63" event={"ID":"7587d6d7-3f38-42d5-98c1-20b6802b09fa","Type":"ContainerStarted","Data":"967381f1d5469ab3700a5a0a4e97ca5d7eb95c52c52919c86c18509a21e59958"}
Sep 09 21:45:49.789188 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:49.789185    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7504-f47c63" event={"ID":"7587d6d7-3f38-42d5-98c1-20b6802b09fa","Type":"ContainerStarted","Data":"740a1b5117e3400735df07c2c16b16171cb5246ca18a49a040914a82bac78ccb"}
Sep 09 21:45:49.789740 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:49.789718    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:49.790467 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:49.790449    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:45:49.807833 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:49.807789    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7504-f47c63" podStartSLOduration=0.93838057 podStartE2EDuration="1.807774704s" totalImagesPullingTime="869.394134ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:45:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:45:49.806336795 +0000 UTC m=+769.279301705" watchObservedRunningTime="2026-09-09 21:45:49.807774704 +0000 UTC m=+769.280739612"
Sep 09 21:45:50.193713 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:50.193677    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:45:50.442197 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:50.442167    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7504-f47c63"]
Sep 09 21:45:50.446845 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:50.446821    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7504-f47c63"]
Sep 09 21:45:50.560613 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:50.560587    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7504-3ac4dd"]
Sep 09 21:45:50.561028 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:50.561014    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7587d6d7-3f38-42d5-98c1-20b6802b09fa" containerName="test-curl-7504-f47c63"
Sep 09 21:45:50.561083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:50.561030    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7587d6d7-3f38-42d5-98c1-20b6802b09fa" containerName="test-curl-7504-f47c63"
Sep 09 21:45:50.561119 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:50.561113    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7587d6d7-3f38-42d5-98c1-20b6802b09fa" containerName="test-curl-7504-f47c63"
Sep 09 21:45:50.563165 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:50.563151    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7504-3ac4dd"
Sep 09 21:45:50.570716 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:50.570692    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7504-3ac4dd"]
Sep 09 21:45:50.653481 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:50.653445    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd64j\" (UniqueName: \"kubernetes.io/projected/f2fca6e0-d1e2-4031-851d-11e22376cc68-kube-api-access-sd64j\") pod \"test-curl-7504-3ac4dd\" (UID: \"f2fca6e0-d1e2-4031-851d-11e22376cc68\") " pod="opendatahub/test-curl-7504-3ac4dd"
Sep 09 21:45:50.759369 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:50.759269    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd64j\" (UniqueName: \"kubernetes.io/projected/f2fca6e0-d1e2-4031-851d-11e22376cc68-kube-api-access-sd64j\") pod \"test-curl-7504-3ac4dd\" (UID: \"f2fca6e0-d1e2-4031-851d-11e22376cc68\") " pod="opendatahub/test-curl-7504-3ac4dd"
Sep 09 21:45:50.767503 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:50.767473    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd64j\" (UniqueName: \"kubernetes.io/projected/f2fca6e0-d1e2-4031-851d-11e22376cc68-kube-api-access-sd64j\") pod \"test-curl-7504-3ac4dd\" (UID: \"f2fca6e0-d1e2-4031-851d-11e22376cc68\") " pod="opendatahub/test-curl-7504-3ac4dd"
Sep 09 21:45:50.795863 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:45:50.795832    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7504-f47c63" container="test-curl-7504-f47c63"
Sep 09 21:45:50.873639 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:50.873617    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7504-3ac4dd"
Sep 09 21:45:51.157598 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:51.157549    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7504-3ac4dd"]
Sep 09 21:45:51.160416 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:45:51.160097    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2fca6e0_d1e2_4031_851d_11e22376cc68.slice/crio-9df8dcac6c3f25f5a697b5a34231fd371683eaa36b35751a6819e243e983401e WatchSource:0}: Error finding container 9df8dcac6c3f25f5a697b5a34231fd371683eaa36b35751a6819e243e983401e: Status 404 returned error can't find the container with id 9df8dcac6c3f25f5a697b5a34231fd371683eaa36b35751a6819e243e983401e
Sep 09 21:45:51.802668 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:51.802625    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7504-3ac4dd" event={"ID":"f2fca6e0-d1e2-4031-851d-11e22376cc68","Type":"ContainerStarted","Data":"81893f7c0402e95558779964c7d3b1a5547ff1efce5677482e838969b29db54f"}
Sep 09 21:45:51.803129 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:51.802680    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7504-3ac4dd" event={"ID":"f2fca6e0-d1e2-4031-851d-11e22376cc68","Type":"ContainerStarted","Data":"9df8dcac6c3f25f5a697b5a34231fd371683eaa36b35751a6819e243e983401e"}
Sep 09 21:45:51.803129 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:51.802727    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7504-f47c63" podUID="7587d6d7-3f38-42d5-98c1-20b6802b09fa" containerName="test-curl-7504-f47c63" containerID="cri-o://967381f1d5469ab3700a5a0a4e97ca5d7eb95c52c52919c86c18509a21e59958" gracePeriod=2
Sep 09 21:45:51.824114 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:51.824060    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7504-3ac4dd" podStartSLOduration=1.8240424530000001 podStartE2EDuration="1.824042453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:45:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:45:51.823912528 +0000 UTC m=+771.296877438" watchObservedRunningTime="2026-09-09 21:45:51.824042453 +0000 UTC m=+771.297007362"
Sep 09 21:45:52.419075 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:52.419038    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7504-3ac4dd"]
Sep 09 21:45:52.421996 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:52.421951    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7504-3ac4dd"]
Sep 09 21:45:52.809545 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:45:52.809516    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7504-3ac4dd" container="test-curl-7504-3ac4dd"
Sep 09 21:45:52.982008 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:52.981960    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"]
Sep 09 21:45:52.982579 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:52.982556    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2fca6e0-d1e2-4031-851d-11e22376cc68" containerName="test-curl-7504-3ac4dd"
Sep 09 21:45:52.982579 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:52.982579    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2fca6e0-d1e2-4031-851d-11e22376cc68" containerName="test-curl-7504-3ac4dd"
Sep 09 21:45:52.982729 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:52.982680    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2fca6e0-d1e2-4031-851d-11e22376cc68" containerName="test-curl-7504-3ac4dd"
Sep 09 21:45:52.985151 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:52.985129    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:52.987303 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:52.987279    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-2aaab1-serving-cert\""
Sep 09 21:45:52.988306 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:52.988285    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 21:45:52.988432 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:52.988300    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qxw5s\""
Sep 09 21:45:52.988432 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:52.988318    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 21:45:52.988557 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:52.988349    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 21:45:53.005239 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.005216    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"]
Sep 09 21:45:53.080536 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.080475    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-maas-api-tls\") pod \"maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:53.080536 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.080507    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-metrics-certs\") pod \"maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:53.080674 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.080545    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn4nx\" (UniqueName: \"kubernetes.io/projected/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-kube-api-access-mn4nx\") pod \"maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:53.080674 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.080646    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:53.181593 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.181557    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn4nx\" (UniqueName: \"kubernetes.io/projected/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-kube-api-access-mn4nx\") pod \"maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:53.181782 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.181635    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:53.181782 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.181697    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-maas-api-tls\") pod \"maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:53.181782 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.181728    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-metrics-certs\") pod \"maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:53.181959 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:45:53.181864    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-serving-cert: secret "maas-api-e2e-inf-a-2aaab1-serving-cert" not found
Sep 09 21:45:53.181959 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:45:53.181945    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-maas-api-tls podName:adec3dcb-a12f-4d93-83df-dac8b9ebeab2 nodeName:}" failed. No retries permitted until 2026-09-09 21:45:53.681924751 +0000 UTC m=+773.154889659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-maas-api-tls") pod "maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth" (UID: "adec3dcb-a12f-4d93-83df-dac8b9ebeab2") : secret "maas-api-e2e-inf-a-2aaab1-serving-cert" not found
Sep 09 21:45:53.182451 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.182424    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:53.184310 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.184279    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-metrics-certs\") pod \"maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:53.193699 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.193674    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn4nx\" (UniqueName: \"kubernetes.io/projected/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-kube-api-access-mn4nx\") pod \"maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:53.205821 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.205797    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"]
Sep 09 21:45:53.208859 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.208838    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.211476 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.211451    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-66cedd-openshift-default-dockercfg-djqsg\""
Sep 09 21:45:53.225796 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.225767    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"]
Sep 09 21:45:53.384171 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.384085    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-credential-socket\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.384171 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.384133    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-workload-socket\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.384171 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.384154    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-podinfo\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.384426 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.384204    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-envoy\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.384426 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.384220    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-workload-certs\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.384426 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.384271    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-token\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.384426 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.384294    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-data\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.384426 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.384310    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td2bh\" (UniqueName: \"kubernetes.io/projected/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-kube-api-access-td2bh\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.384426 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.384409    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istiod-ca-cert\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.485692 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.485657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-data\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.485692 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.485692    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td2bh\" (UniqueName: \"kubernetes.io/projected/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-kube-api-access-td2bh\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.485887 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.485875    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istiod-ca-cert\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.485960 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.485947    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-credential-socket\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.486040 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.486006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-workload-socket\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.486040 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.486026    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-podinfo\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.486140 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.486063    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-envoy\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.486140 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.486065    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-data\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.486140 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.486081    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-workload-certs\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.486333 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.486158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-token\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.486416 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.486393    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-credential-socket\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.486530 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.486509    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-workload-socket\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.486575 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.486537    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-workload-certs\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.486650 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.486626    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istiod-ca-cert\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.488306 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.488284    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-envoy\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.488428 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.488408    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-podinfo\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.493746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.493715    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td2bh\" (UniqueName: \"kubernetes.io/projected/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-kube-api-access-td2bh\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.493826 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.493747    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-token\") pod \"e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") " pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.526483 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.526464    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:53.688139 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.688059    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-maas-api-tls\") pod \"maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:53.690456 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.690432    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-maas-api-tls\") pod \"maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:53.819753 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.819693    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"]
Sep 09 21:45:53.822081 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.821703    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7504-3ac4dd" podUID="f2fca6e0-d1e2-4031-851d-11e22376cc68" containerName="test-curl-7504-3ac4dd" containerID="cri-o://81893f7c0402e95558779964c7d3b1a5547ff1efce5677482e838969b29db54f" gracePeriod=2
Sep 09 21:45:53.875029 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:45:53.874998    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f7deee2_30a8_4ac4_9b6f_12cdb883e761.slice/crio-da46ff4cc908a4e0266e53137b1edc93e20018bd056a92e9f436c0da633283a5 WatchSource:0}: Error finding container da46ff4cc908a4e0266e53137b1edc93e20018bd056a92e9f436c0da633283a5: Status 404 returned error can't find the container with id da46ff4cc908a4e0266e53137b1edc93e20018bd056a92e9f436c0da633283a5
Sep 09 21:45:53.877728 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.877688    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:45:53.877832 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.877769    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:45:53.877832 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.877803    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:45:53.896681 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.896586    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:53.993782 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:53.993605    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7504-f47c63"
Sep 09 21:45:54.091593 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:54.091558    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8th5\" (UniqueName: \"kubernetes.io/projected/7587d6d7-3f38-42d5-98c1-20b6802b09fa-kube-api-access-x8th5\") pod \"7587d6d7-3f38-42d5-98c1-20b6802b09fa\" (UID: \"7587d6d7-3f38-42d5-98c1-20b6802b09fa\") "
Sep 09 21:45:54.093744 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:54.093718    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7587d6d7-3f38-42d5-98c1-20b6802b09fa-kube-api-access-x8th5" (OuterVolumeSpecName: "kube-api-access-x8th5") pod "7587d6d7-3f38-42d5-98c1-20b6802b09fa" (UID: "7587d6d7-3f38-42d5-98c1-20b6802b09fa"). InnerVolumeSpecName "kube-api-access-x8th5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:45:54.176205 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:54.176178    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"]
Sep 09 21:45:54.177067 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:45:54.177045    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadec3dcb_a12f_4d93_83df_dac8b9ebeab2.slice/crio-5f14f98f2737532aa4afb4b4df975c6842338a314e2050abce97f28812715e1a WatchSource:0}: Error finding container 5f14f98f2737532aa4afb4b4df975c6842338a314e2050abce97f28812715e1a: Status 404 returned error can't find the container with id 5f14f98f2737532aa4afb4b4df975c6842338a314e2050abce97f28812715e1a
Sep 09 21:45:54.192548 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:54.192517    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8th5\" (UniqueName: \"kubernetes.io/projected/7587d6d7-3f38-42d5-98c1-20b6802b09fa-kube-api-access-x8th5\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:45:54.832403 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:54.832233    2578 generic.go:356] "Generic (PLEG): container finished" podID="7587d6d7-3f38-42d5-98c1-20b6802b09fa" containerID="967381f1d5469ab3700a5a0a4e97ca5d7eb95c52c52919c86c18509a21e59958" exitCode=137
Sep 09 21:45:54.832403 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:54.832283    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7504-f47c63"
Sep 09 21:45:54.832403 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:54.832311    2578 scope.go:117] "RemoveContainer" containerID="967381f1d5469ab3700a5a0a4e97ca5d7eb95c52c52919c86c18509a21e59958"
Sep 09 21:45:54.834568 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:54.834479    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth" event={"ID":"adec3dcb-a12f-4d93-83df-dac8b9ebeab2","Type":"ContainerStarted","Data":"5f14f98f2737532aa4afb4b4df975c6842338a314e2050abce97f28812715e1a"}
Sep 09 21:45:54.836905 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:54.836725    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk" event={"ID":"8f7deee2-30a8-4ac4-9b6f-12cdb883e761","Type":"ContainerStarted","Data":"7c582e740c5995abdb150b22566f02192b75d30ef3d89462567943316c64f38a"}
Sep 09 21:45:54.836905 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:54.836784    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk" event={"ID":"8f7deee2-30a8-4ac4-9b6f-12cdb883e761","Type":"ContainerStarted","Data":"da46ff4cc908a4e0266e53137b1edc93e20018bd056a92e9f436c0da633283a5"}
Sep 09 21:45:54.847231 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:54.847211    2578 scope.go:117] "RemoveContainer" containerID="967381f1d5469ab3700a5a0a4e97ca5d7eb95c52c52919c86c18509a21e59958"
Sep 09 21:45:54.847665 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:45:54.847644    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"967381f1d5469ab3700a5a0a4e97ca5d7eb95c52c52919c86c18509a21e59958\": container with ID starting with 967381f1d5469ab3700a5a0a4e97ca5d7eb95c52c52919c86c18509a21e59958 not found: ID does not exist" containerID="967381f1d5469ab3700a5a0a4e97ca5d7eb95c52c52919c86c18509a21e59958"
Sep 09 21:45:54.847750 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:54.847676    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"967381f1d5469ab3700a5a0a4e97ca5d7eb95c52c52919c86c18509a21e59958"} err="failed to get container status \"967381f1d5469ab3700a5a0a4e97ca5d7eb95c52c52919c86c18509a21e59958\": rpc error: code = NotFound desc = could not find container \"967381f1d5469ab3700a5a0a4e97ca5d7eb95c52c52919c86c18509a21e59958\": container with ID starting with 967381f1d5469ab3700a5a0a4e97ca5d7eb95c52c52919c86c18509a21e59958 not found: ID does not exist"
Sep 09 21:45:54.862423 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:54.862359    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk" podStartSLOduration=1.862343207 podStartE2EDuration="1.862343207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:45:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:45:54.861411812 +0000 UTC m=+774.334376723" watchObservedRunningTime="2026-09-09 21:45:54.862343207 +0000 UTC m=+774.335308117"
Sep 09 21:45:54.863077 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:45:54.863041    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7504-f47c63\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="7587d6d7-3f38-42d5-98c1-20b6802b09fa" pod="opendatahub/test-curl-7504-f47c63"
Sep 09 21:45:55.007316 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.007282    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7587d6d7-3f38-42d5-98c1-20b6802b09fa" path="/var/lib/kubelet/pods/7587d6d7-3f38-42d5-98c1-20b6802b09fa/volumes"
Sep 09 21:45:55.527142 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.527109    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:55.533282 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.533251    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:55.605225 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.605193    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"]
Sep 09 21:45:55.611816 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.611786    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.614163 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.614110    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-04d63732-openshift-default-dockercfg-vhgdh\""
Sep 09 21:45:55.620111 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.620088    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"]
Sep 09 21:45:55.707314 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.707280    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2ghm\" (UniqueName: \"kubernetes.io/projected/8411ae49-a386-4f6d-91ca-9367219465fb-kube-api-access-z2ghm\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.707518 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.707326    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-istio-envoy\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.707518 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.707352    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8411ae49-a386-4f6d-91ca-9367219465fb-istio-podinfo\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.707518 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.707421    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8411ae49-a386-4f6d-91ca-9367219465fb-istiod-ca-cert\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.707518 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.707455    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-workload-certs\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.707518 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.707492    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-workload-socket\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.707777 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.707533    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-istio-data\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.707777 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.707566    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-credential-socket\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.707777 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.707626    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8411ae49-a386-4f6d-91ca-9367219465fb-istio-token\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.808490 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.808400    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8411ae49-a386-4f6d-91ca-9367219465fb-istio-podinfo\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.808656 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.808507    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8411ae49-a386-4f6d-91ca-9367219465fb-istiod-ca-cert\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.808656 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.808546    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-workload-certs\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.808780 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.808668    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-workload-socket\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.808780 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.808729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-istio-data\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.808780 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.808765    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-credential-socket\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.808943 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.808826    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8411ae49-a386-4f6d-91ca-9367219465fb-istio-token\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.808943 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.808899    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2ghm\" (UniqueName: \"kubernetes.io/projected/8411ae49-a386-4f6d-91ca-9367219465fb-kube-api-access-z2ghm\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.808943 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.808901    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-workload-certs\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.808943 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.808929    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-istio-envoy\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.809310 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.809222    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-istio-data\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.809443 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.809307    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8411ae49-a386-4f6d-91ca-9367219465fb-istiod-ca-cert\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.809443 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.809408    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-workload-socket\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.809577 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.809537    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-credential-socket\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.811451 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.811430    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8411ae49-a386-4f6d-91ca-9367219465fb-istio-podinfo\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.811556 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.811451    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-istio-envoy\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.817221 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.817198    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8411ae49-a386-4f6d-91ca-9367219465fb-istio-token\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.817391 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.817374    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2ghm\" (UniqueName: \"kubernetes.io/projected/8411ae49-a386-4f6d-91ca-9367219465fb-kube-api-access-z2ghm\") pod \"e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") " pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:55.845215 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.845191    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:55.846304 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.846284    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:45:55.926463 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:55.926433    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:56.583122 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.583091    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7504-3ac4dd"
Sep 09 21:45:56.719074 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.719043    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sd64j\" (UniqueName: \"kubernetes.io/projected/f2fca6e0-d1e2-4031-851d-11e22376cc68-kube-api-access-sd64j\") pod \"f2fca6e0-d1e2-4031-851d-11e22376cc68\" (UID: \"f2fca6e0-d1e2-4031-851d-11e22376cc68\") "
Sep 09 21:45:56.721119 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.721094    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2fca6e0-d1e2-4031-851d-11e22376cc68-kube-api-access-sd64j" (OuterVolumeSpecName: "kube-api-access-sd64j") pod "f2fca6e0-d1e2-4031-851d-11e22376cc68" (UID: "f2fca6e0-d1e2-4031-851d-11e22376cc68"). InnerVolumeSpecName "kube-api-access-sd64j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:45:56.820980 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.820938    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sd64j\" (UniqueName: \"kubernetes.io/projected/f2fca6e0-d1e2-4031-851d-11e22376cc68-kube-api-access-sd64j\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:45:56.836326 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.836295    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"]
Sep 09 21:45:56.837285 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:45:56.837257    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8411ae49_a386_4f6d_91ca_9367219465fb.slice/crio-38a0e2bd15a459f77b48fcc1b4e6d44f38d458704096943283551230a301b5b1 WatchSource:0}: Error finding container 38a0e2bd15a459f77b48fcc1b4e6d44f38d458704096943283551230a301b5b1: Status 404 returned error can't find the container with id 38a0e2bd15a459f77b48fcc1b4e6d44f38d458704096943283551230a301b5b1
Sep 09 21:45:56.839995 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.839937    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:45:56.840077 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.840043    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:45:56.840152 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.840133    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:45:56.855998 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.855947    2578 generic.go:356] "Generic (PLEG): container finished" podID="f2fca6e0-d1e2-4031-851d-11e22376cc68" containerID="81893f7c0402e95558779964c7d3b1a5547ff1efce5677482e838969b29db54f" exitCode=137
Sep 09 21:45:56.856372 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.856018    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7504-3ac4dd"
Sep 09 21:45:56.856372 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.856085    2578 scope.go:117] "RemoveContainer" containerID="81893f7c0402e95558779964c7d3b1a5547ff1efce5677482e838969b29db54f"
Sep 09 21:45:56.858057 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.858021    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth" event={"ID":"adec3dcb-a12f-4d93-83df-dac8b9ebeab2","Type":"ContainerStarted","Data":"2e13280c1c2fcd2d746d245b8dcb88abc0255bfc60f8b79f26f01592998540a6"}
Sep 09 21:45:56.858147 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.858115    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:45:56.859577 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.859550    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg" event={"ID":"8411ae49-a386-4f6d-91ca-9367219465fb","Type":"ContainerStarted","Data":"38a0e2bd15a459f77b48fcc1b4e6d44f38d458704096943283551230a301b5b1"}
Sep 09 21:45:56.867296 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.867273    2578 scope.go:117] "RemoveContainer" containerID="81893f7c0402e95558779964c7d3b1a5547ff1efce5677482e838969b29db54f"
Sep 09 21:45:56.867634 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:45:56.867607    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81893f7c0402e95558779964c7d3b1a5547ff1efce5677482e838969b29db54f\": container with ID starting with 81893f7c0402e95558779964c7d3b1a5547ff1efce5677482e838969b29db54f not found: ID does not exist" containerID="81893f7c0402e95558779964c7d3b1a5547ff1efce5677482e838969b29db54f"
Sep 09 21:45:56.867704 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.867643    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81893f7c0402e95558779964c7d3b1a5547ff1efce5677482e838969b29db54f"} err="failed to get container status \"81893f7c0402e95558779964c7d3b1a5547ff1efce5677482e838969b29db54f\": rpc error: code = NotFound desc = could not find container \"81893f7c0402e95558779964c7d3b1a5547ff1efce5677482e838969b29db54f\": container with ID starting with 81893f7c0402e95558779964c7d3b1a5547ff1efce5677482e838969b29db54f not found: ID does not exist"
Sep 09 21:45:56.880597 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:56.880498    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth" podStartSLOduration=2.441415977 podStartE2EDuration="4.880483346s" totalImagesPullingTime="2.439067369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:45:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:45:56.880107306 +0000 UTC m=+776.353072261" watchObservedRunningTime="2026-09-09 21:45:56.880483346 +0000 UTC m=+776.353448257"
Sep 09 21:45:56.882004 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:45:56.881978    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7504-3ac4dd\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="f2fca6e0-d1e2-4031-851d-11e22376cc68" pod="opendatahub/test-curl-7504-3ac4dd"
Sep 09 21:45:57.005244 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:57.005203    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2fca6e0-d1e2-4031-851d-11e22376cc68" path="/var/lib/kubelet/pods/f2fca6e0-d1e2-4031-851d-11e22376cc68/volumes"
Sep 09 21:45:57.866743 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:57.866705    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg" event={"ID":"8411ae49-a386-4f6d-91ca-9367219465fb","Type":"ContainerStarted","Data":"120b9e4285dd2f9fc4628a944b0e2843ac604500cb8de7f7127081af122e578a"}
Sep 09 21:45:57.885711 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:57.885662    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg" podStartSLOduration=2.885646978 podStartE2EDuration="2.885646978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:45:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:45:57.88411661 +0000 UTC m=+777.357081520" watchObservedRunningTime="2026-09-09 21:45:57.885646978 +0000 UTC m=+777.358611888"
Sep 09 21:45:57.927015 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:57.926982    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:57.935059 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:57.933997    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:58.874790 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:58.874730    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:45:58.875943 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:45:58.875920    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:46:02.879322 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:02.879264    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:46:06.367668 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:06.367628    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:46:11.023119 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.023084    2578 generic.go:356] "Generic (PLEG): container finished" podID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerID="bebe50c6962b4d169194addd2d75de38b300d8d222a6533f2d7c114473113658" exitCode=0
Sep 09 21:46:11.023536 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.023156    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" event={"ID":"787bc763-f469-4eb4-a3cb-bef97d299dd0","Type":"ContainerDied","Data":"bebe50c6962b4d169194addd2d75de38b300d8d222a6533f2d7c114473113658"}
Sep 09 21:46:11.023536 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.023515    2578 scope.go:117] "RemoveContainer" containerID="bebe50c6962b4d169194addd2d75de38b300d8d222a6533f2d7c114473113658"
Sep 09 21:46:11.024697 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.024659    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:11.024787 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.024759    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:11.024829 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.024806    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:11.024944 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.024926    2578 generic.go:356] "Generic (PLEG): container finished" podID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerID="af1f598aba0235e89dd793bb0ff0bca71e507ca350c7804a4a7d464bc1a2f955" exitCode=0
Sep 09 21:46:11.025024 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.024997    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb" event={"ID":"06282b23-1694-4fc2-b3c3-c17e924cca00","Type":"ContainerDied","Data":"af1f598aba0235e89dd793bb0ff0bca71e507ca350c7804a4a7d464bc1a2f955"}
Sep 09 21:46:11.025397 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.025379    2578 scope.go:117] "RemoveContainer" containerID="af1f598aba0235e89dd793bb0ff0bca71e507ca350c7804a4a7d464bc1a2f955"
Sep 09 21:46:11.026594 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.026400    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:11.026594 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.026466    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:11.026594 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.026502    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:11.026762 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.026708    2578 generic.go:356] "Generic (PLEG): container finished" podID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerID="00b8c89d9eea7e7fe3309f7de78726d9a8cfcc95eb1066167ef353f02f740615" exitCode=0
Sep 09 21:46:11.026762 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.026756    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" event={"ID":"40c04813-7e10-4669-a048-93ad3f50c5b4","Type":"ContainerDied","Data":"00b8c89d9eea7e7fe3309f7de78726d9a8cfcc95eb1066167ef353f02f740615"}
Sep 09 21:46:11.027348 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.027096    2578 scope.go:117] "RemoveContainer" containerID="00b8c89d9eea7e7fe3309f7de78726d9a8cfcc95eb1066167ef353f02f740615"
Sep 09 21:46:11.028414 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.028328    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:11.028474 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.028438    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:11.028510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:11.028482    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:12.040212 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:12.039170    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" event={"ID":"787bc763-f469-4eb4-a3cb-bef97d299dd0","Type":"ContainerStarted","Data":"4a2c4ca5e012bf4c0c3098adf82f0c01778f6df30a6a3c404b0d35ec796f6632"}
Sep 09 21:46:12.045082 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:12.045038    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb" event={"ID":"06282b23-1694-4fc2-b3c3-c17e924cca00","Type":"ContainerStarted","Data":"16475e0352609543452b124a4979663dd977f88b3bba7bebcaaa9378b7136ac7"}
Sep 09 21:46:12.048226 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:12.048180    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" event={"ID":"40c04813-7e10-4669-a048-93ad3f50c5b4","Type":"ContainerStarted","Data":"00619049f7a84b437a89024acb21b0ceda0f98925c8eaa479c2fe743ad8a29a4"}
Sep 09 21:46:12.116764 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:12.116721    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:46:12.116992 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:12.116954    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 09 21:46:12.117064 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:12.117023    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb" podUID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 09 21:46:12.194229 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:12.194193    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:46:12.194876 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:12.194788    2578 patch_prober.go:28] interesting pod/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 09 21:46:12.195132 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:12.195104    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 09 21:46:12.979065 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:12.979012    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:46:12.979296 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:12.979155    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 09 21:46:12.979296 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:12.979199    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 09 21:46:13.117821 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:13.117768    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 09 21:46:13.118372 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:13.117843    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb" podUID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 09 21:46:13.194233 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:13.194188    2578 patch_prober.go:28] interesting pod/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 09 21:46:13.194490 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:13.194258    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 09 21:46:13.993024 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:13.990543    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 09 21:46:13.993024 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:13.990613    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 09 21:46:14.117991 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:14.117458    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 09 21:46:14.117991 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:14.117534    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb" podUID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 09 21:46:14.194243 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:14.194199    2578 patch_prober.go:28] interesting pod/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 09 21:46:14.194428 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:14.194282    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 09 21:46:14.979456 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:14.979392    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 09 21:46:14.979653 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:14.979461    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 09 21:46:15.117033 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:15.116995    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 09 21:46:15.117298 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:15.117272    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb" podUID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 09 21:46:15.194598 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:15.194564    2578 patch_prober.go:28] interesting pod/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 09 21:46:15.195212 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:15.195166    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 09 21:46:15.979876 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:15.979825    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 09 21:46:15.980107 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:15.979900    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 09 21:46:16.116872 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:16.116826    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 09 21:46:16.117085 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:16.116915    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb" podUID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 09 21:46:16.193658 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:16.193614    2578 patch_prober.go:28] interesting pod/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 09 21:46:16.193862 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:16.193688    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 09 21:46:16.980232 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:16.979754    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 09 21:46:16.980232 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:16.979829    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 09 21:46:17.119409 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:17.119364    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 09 21:46:17.119573 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:17.119430    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb" podUID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 09 21:46:17.199992 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:17.194020    2578 patch_prober.go:28] interesting pod/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 09 21:46:17.199992 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:17.194084    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 09 21:46:17.979243 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:17.979140    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 09 21:46:17.980815 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:17.979501    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 09 21:46:18.117454 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:18.117414    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 09 21:46:18.117683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:18.117664    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb" podUID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 09 21:46:18.194267 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:18.194233    2578 patch_prober.go:28] interesting pod/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 09 21:46:18.194449 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:18.194295    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 09 21:46:18.782458 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:18.782420    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:46:18.782630 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:18.782507    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:46:18.978958 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:18.978915    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 09 21:46:18.979186 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:18.978998    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 09 21:46:19.122637 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:19.122538    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:46:19.129190 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:19.129162    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:46:19.200078 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:19.200039    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:46:19.201536 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:19.201502    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:46:19.789665 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:19.789634    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:46:19.983679 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:19.983592    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:46:20.138244 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.138214    2578 generic.go:356] "Generic (PLEG): container finished" podID="8f7deee2-30a8-4ac4-9b6f-12cdb883e761" containerID="7c582e740c5995abdb150b22566f02192b75d30ef3d89462567943316c64f38a" exitCode=0
Sep 09 21:46:20.139773 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.138284    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk" event={"ID":"8f7deee2-30a8-4ac4-9b6f-12cdb883e761","Type":"ContainerDied","Data":"7c582e740c5995abdb150b22566f02192b75d30ef3d89462567943316c64f38a"}
Sep 09 21:46:20.139773 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.139224    2578 scope.go:117] "RemoveContainer" containerID="7c582e740c5995abdb150b22566f02192b75d30ef3d89462567943316c64f38a"
Sep 09 21:46:20.139773 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.139596    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 21:46:20.140496 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.140464    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:20.140601 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.140548    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:20.140669 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.140596    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:20.350172 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.350102    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"]
Sep 09 21:46:20.353386 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.353364    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.356029 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.356008    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-da0a18-openshift-default-dockercfg-mpgn7\""
Sep 09 21:46:20.367306 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.367280    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"]
Sep 09 21:46:20.438416 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.438382    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-workload-socket\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.438636 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.438437    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-workload-certs\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.438636 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.438495    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqc9s\" (UniqueName: \"kubernetes.io/projected/e5ca4200-2da3-459c-9feb-deeb311d1d70-kube-api-access-fqc9s\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.438636 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.438548    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5ca4200-2da3-459c-9feb-deeb311d1d70-istiod-ca-cert\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.438636 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.438580    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-data\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.438821 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.438656    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-podinfo\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.438821 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.438693    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-credential-socket\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.438821 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.438774    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-token\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.438821 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.438801    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-envoy\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.539647 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.539614    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-workload-socket\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.539849 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.539676    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-workload-certs\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.539849 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.539710    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqc9s\" (UniqueName: \"kubernetes.io/projected/e5ca4200-2da3-459c-9feb-deeb311d1d70-kube-api-access-fqc9s\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.539849 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.539743    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5ca4200-2da3-459c-9feb-deeb311d1d70-istiod-ca-cert\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.539849 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.539770    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-data\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.539849 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.539809    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-podinfo\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.539849 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.539846    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-credential-socket\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.540192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.539940    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-token\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.540192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.540119    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-envoy\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.540192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.540182    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-workload-certs\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.540355 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.540224    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-credential-socket\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.540355 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.540334    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-workload-socket\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.540497 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.540474    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-data\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.540722 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.540693    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5ca4200-2da3-459c-9feb-deeb311d1d70-istiod-ca-cert\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.542591 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.542570    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-envoy\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.542902 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.542882    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-podinfo\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.549361 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.549335    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-token\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.549563 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.549546    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqc9s\" (UniqueName: \"kubernetes.io/projected/e5ca4200-2da3-459c-9feb-deeb311d1d70-kube-api-access-fqc9s\") pod \"e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") " pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.665952 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.665898    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:20.950019 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.949958    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"]
Sep 09 21:46:20.957042 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:46:20.956085    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5ca4200_2da3_459c_9feb_deeb311d1d70.slice/crio-066219fd261c5d51ae17e949e534059d79b49ebf34f8e6963bd7f4cb796dbdcc WatchSource:0}: Error finding container 066219fd261c5d51ae17e949e534059d79b49ebf34f8e6963bd7f4cb796dbdcc: Status 404 returned error can't find the container with id 066219fd261c5d51ae17e949e534059d79b49ebf34f8e6963bd7f4cb796dbdcc
Sep 09 21:46:20.959674 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.958900    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:20.959674 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.959005    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:20.959674 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:20.959051    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:21.154494 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:21.154458    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk" event={"ID":"8f7deee2-30a8-4ac4-9b6f-12cdb883e761","Type":"ContainerStarted","Data":"a835c9cadb65f4056cc2f2e266d52d4371dbfb0c313afbea657816fce0f1f337"}
Sep 09 21:46:21.157921 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:21.157891    2578 generic.go:356] "Generic (PLEG): container finished" podID="8411ae49-a386-4f6d-91ca-9367219465fb" containerID="120b9e4285dd2f9fc4628a944b0e2843ac604500cb8de7f7127081af122e578a" exitCode=0
Sep 09 21:46:21.158229 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:21.158206    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg" event={"ID":"8411ae49-a386-4f6d-91ca-9367219465fb","Type":"ContainerDied","Data":"120b9e4285dd2f9fc4628a944b0e2843ac604500cb8de7f7127081af122e578a"}
Sep 09 21:46:21.160497 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:21.160478    2578 scope.go:117] "RemoveContainer" containerID="120b9e4285dd2f9fc4628a944b0e2843ac604500cb8de7f7127081af122e578a"
Sep 09 21:46:21.161728 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:21.161698    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5" event={"ID":"e5ca4200-2da3-459c-9feb-deeb311d1d70","Type":"ContainerStarted","Data":"07dc140b9ab31425fd4232aab3a6e1ba469681ecbf6bd0b38d2736c48e95eb9f"}
Sep 09 21:46:21.161831 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:21.161749    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5" event={"ID":"e5ca4200-2da3-459c-9feb-deeb311d1d70","Type":"ContainerStarted","Data":"066219fd261c5d51ae17e949e534059d79b49ebf34f8e6963bd7f4cb796dbdcc"}
Sep 09 21:46:21.162792 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:21.162756    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:21.162883 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:21.162823    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:21.162883 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:21.162857    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:21.250987 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:21.246317    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5" podStartSLOduration=1.246287421 podStartE2EDuration="1.246287421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:46:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:46:21.228705987 +0000 UTC m=+800.701670893" watchObservedRunningTime="2026-09-09 21:46:21.246287421 +0000 UTC m=+800.719252536"
Sep 09 21:46:21.527243 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:21.527213    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:46:21.528420 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:21.528394    2578 patch_prober.go:28] interesting pod/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 09 21:46:21.528558 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:21.528460    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk" podUID="8f7deee2-30a8-4ac4-9b6f-12cdb883e761" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 09 21:46:21.666861 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:21.666820    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:22.174781 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:22.174745    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg" event={"ID":"8411ae49-a386-4f6d-91ca-9367219465fb","Type":"ContainerStarted","Data":"b4f6513c551aaf9abff1452c5a79c17e0aee01879f4b7fcd960c76fa3fc7ae8a"}
Sep 09 21:46:22.527316 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:22.527281    2578 patch_prober.go:28] interesting pod/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 09 21:46:22.527492 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:22.527344    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk" podUID="8f7deee2-30a8-4ac4-9b6f-12cdb883e761" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 09 21:46:22.676014 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:22.675980    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:22.927698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:22.927608    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:46:22.927938 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:22.927917    2578 patch_prober.go:28] interesting pod/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 09 21:46:22.928068 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:22.928018    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg" podUID="8411ae49-a386-4f6d-91ca-9367219465fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 09 21:46:23.185592 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:23.185504    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:23.187819 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:23.187799    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:23.527607 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:23.527579    2578 patch_prober.go:28] interesting pod/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 09 21:46:23.527770 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:23.527644    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk" podUID="8f7deee2-30a8-4ac4-9b6f-12cdb883e761" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 09 21:46:23.874177 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:23.873920    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:46:23.927270 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:23.927223    2578 patch_prober.go:28] interesting pod/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 09 21:46:23.927437 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:23.927292    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg" podUID="8411ae49-a386-4f6d-91ca-9367219465fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 09 21:46:24.527276 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:24.527239    2578 patch_prober.go:28] interesting pod/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 09 21:46:24.527698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:24.527289    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk" podUID="8f7deee2-30a8-4ac4-9b6f-12cdb883e761" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 09 21:46:24.927901 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:24.927829    2578 patch_prober.go:28] interesting pod/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 09 21:46:24.928072 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:24.928044    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg" podUID="8411ae49-a386-4f6d-91ca-9367219465fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 09 21:46:25.527843 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:25.527808    2578 patch_prober.go:28] interesting pod/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 09 21:46:25.528343 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:25.527871    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk" podUID="8f7deee2-30a8-4ac4-9b6f-12cdb883e761" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 09 21:46:25.846155 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:25.846061    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:46:25.927851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:25.927814    2578 patch_prober.go:28] interesting pod/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 09 21:46:25.928040 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:25.927866    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg" podUID="8411ae49-a386-4f6d-91ca-9367219465fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 09 21:46:26.530836 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:26.530806    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:46:26.536581 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:26.536551    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:46:26.927866 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:26.927788    2578 patch_prober.go:28] interesting pod/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 09 21:46:26.927866 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:26.927843    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg" podUID="8411ae49-a386-4f6d-91ca-9367219465fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 09 21:46:27.931254 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:27.931222    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:46:27.931747 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:27.931736    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:46:27.932552 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:27.932532    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:46:38.066323 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:38.066291    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:46:41.877409 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:41.877366    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"]
Sep 09 21:46:41.878371 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:41.878312    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg" podUID="8411ae49-a386-4f6d-91ca-9367219465fb" containerName="istio-proxy" containerID="cri-o://b4f6513c551aaf9abff1452c5a79c17e0aee01879f4b7fcd960c76fa3fc7ae8a" gracePeriod=30
Sep 09 21:46:42.842558 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:42.842528    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"]
Sep 09 21:46:42.845253 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:42.845232    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:42.848673 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:42.848537    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-98c7fb47-gw-openshift-default-dockercfg-vdszp\""
Sep 09 21:46:42.860128 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:42.860104    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"]
Sep 09 21:46:42.933594 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:42.933508    2578 patch_prober.go:28] interesting pod/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 21:46:42.933594 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:42.933577    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg" podUID="8411ae49-a386-4f6d-91ca-9367219465fb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:46:42.995496 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:42.995460    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-envoy\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:42.995691 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:42.995507    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-workload-certs\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:42.995691 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:42.995575    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istiod-ca-cert\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:42.995691 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:42.995641    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt5c7\" (UniqueName: \"kubernetes.io/projected/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-kube-api-access-zt5c7\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:42.995691 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:42.995678    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-data\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:42.995857 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:42.995703    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-workload-socket\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:42.995857 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:42.995730    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-token\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:42.995857 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:42.995824    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-credential-socket\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:42.995955 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:42.995868    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-podinfo\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.096596 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.096525    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-envoy\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.096596 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.096563    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-workload-certs\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.096596 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.096582    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istiod-ca-cert\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.096858 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.096614    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt5c7\" (UniqueName: \"kubernetes.io/projected/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-kube-api-access-zt5c7\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.096858 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.096655    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-data\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.096858 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.096677    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-workload-socket\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.096858 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.096709    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-token\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.096858 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.096760    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-credential-socket\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.096858 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.096794    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-podinfo\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.097212 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.097082    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-workload-certs\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.097303 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.097278    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-data\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.097427 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.097403    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istiod-ca-cert\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.097545 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.097433    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-workload-socket\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.097545 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.097512    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-credential-socket\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.098880 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.098856    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-envoy\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.099308 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.099287    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-podinfo\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.105828 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.105806    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt5c7\" (UniqueName: \"kubernetes.io/projected/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-kube-api-access-zt5c7\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.105906 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.105868    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-token\") pod \"e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") " pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.158642 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.158608    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:43.324847 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.324818    2578 generic.go:356] "Generic (PLEG): container finished" podID="e5ca4200-2da3-459c-9feb-deeb311d1d70" containerID="07dc140b9ab31425fd4232aab3a6e1ba469681ecbf6bd0b38d2736c48e95eb9f" exitCode=0
Sep 09 21:46:43.325077 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.324861    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5" event={"ID":"e5ca4200-2da3-459c-9feb-deeb311d1d70","Type":"ContainerDied","Data":"07dc140b9ab31425fd4232aab3a6e1ba469681ecbf6bd0b38d2736c48e95eb9f"}
Sep 09 21:46:43.325341 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.325321    2578 scope.go:117] "RemoveContainer" containerID="07dc140b9ab31425fd4232aab3a6e1ba469681ecbf6bd0b38d2736c48e95eb9f"
Sep 09 21:46:43.326729 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.326695    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:43.326823 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.326768    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:43.326823 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.326801    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:43.436499 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.436461    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"]
Sep 09 21:46:43.437792 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:46:43.437762    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb2f107_6cb3_47f6_8bbf_14ce82818c19.slice/crio-e6c403c15e3c773b797f7794e8138bdad2f269ae076bf2ba7ac7d46295127914 WatchSource:0}: Error finding container e6c403c15e3c773b797f7794e8138bdad2f269ae076bf2ba7ac7d46295127914: Status 404 returned error can't find the container with id e6c403c15e3c773b797f7794e8138bdad2f269ae076bf2ba7ac7d46295127914
Sep 09 21:46:43.440666 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.440614    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:43.440760 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.440713    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:43.440809 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:43.440760    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:46:44.339660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:44.339618    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5" event={"ID":"e5ca4200-2da3-459c-9feb-deeb311d1d70","Type":"ContainerStarted","Data":"2471b4508d082f5b2aeda930c44f5bade703f32fe8794cb68f1a7071f7cea217"}
Sep 09 21:46:44.341488 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:44.341452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t" event={"ID":"5eb2f107-6cb3-47f6-8bbf-14ce82818c19","Type":"ContainerStarted","Data":"41b1cedaca09fe2d67e982a307575f992c169a9bd165bab221b95e67b5495781"}
Sep 09 21:46:44.341619 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:44.341500    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t" event={"ID":"5eb2f107-6cb3-47f6-8bbf-14ce82818c19","Type":"ContainerStarted","Data":"e6c403c15e3c773b797f7794e8138bdad2f269ae076bf2ba7ac7d46295127914"}
Sep 09 21:46:44.372369 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:44.372329    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t" podStartSLOduration=2.372315617 podStartE2EDuration="2.372315617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:46:44.372049204 +0000 UTC m=+823.845014113" watchObservedRunningTime="2026-09-09 21:46:44.372315617 +0000 UTC m=+823.845280527"
Sep 09 21:46:44.667256 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:44.667165    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:44.667503 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:44.667479    2578 patch_prober.go:28] interesting pod/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 09 21:46:44.667569 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:44.667526    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5" podUID="e5ca4200-2da3-459c-9feb-deeb311d1d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 09 21:46:45.159496 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:45.159455    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:45.165960 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:45.165932    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:45.353253 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:45.353222    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:45.358488 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:45.358457    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:46:45.666577 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:45.666545    2578 patch_prober.go:28] interesting pod/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 09 21:46:45.666765 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:45.666609    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5" podUID="e5ca4200-2da3-459c-9feb-deeb311d1d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 09 21:46:46.666772 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:46.666736    2578 patch_prober.go:28] interesting pod/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 09 21:46:46.667240 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:46.666791    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5" podUID="e5ca4200-2da3-459c-9feb-deeb311d1d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 09 21:46:47.226250 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.225850    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:46:47.355994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.354771    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-istio-envoy\") pod \"8411ae49-a386-4f6d-91ca-9367219465fb\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") "
Sep 09 21:46:47.355994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.354881    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-istio-data\") pod \"8411ae49-a386-4f6d-91ca-9367219465fb\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") "
Sep 09 21:46:47.355994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.354939    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8411ae49-a386-4f6d-91ca-9367219465fb-istio-podinfo\") pod \"8411ae49-a386-4f6d-91ca-9367219465fb\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") "
Sep 09 21:46:47.355994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.354993    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-credential-socket\") pod \"8411ae49-a386-4f6d-91ca-9367219465fb\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") "
Sep 09 21:46:47.355994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.355026    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8411ae49-a386-4f6d-91ca-9367219465fb-istiod-ca-cert\") pod \"8411ae49-a386-4f6d-91ca-9367219465fb\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") "
Sep 09 21:46:47.355994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.355086    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-workload-socket\") pod \"8411ae49-a386-4f6d-91ca-9367219465fb\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") "
Sep 09 21:46:47.355994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.355117    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2ghm\" (UniqueName: \"kubernetes.io/projected/8411ae49-a386-4f6d-91ca-9367219465fb-kube-api-access-z2ghm\") pod \"8411ae49-a386-4f6d-91ca-9367219465fb\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") "
Sep 09 21:46:47.355994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.355254    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8411ae49-a386-4f6d-91ca-9367219465fb-istio-token\") pod \"8411ae49-a386-4f6d-91ca-9367219465fb\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") "
Sep 09 21:46:47.355994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.355324    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-workload-certs\") pod \"8411ae49-a386-4f6d-91ca-9367219465fb\" (UID: \"8411ae49-a386-4f6d-91ca-9367219465fb\") "
Sep 09 21:46:47.356623 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.356071    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8411ae49-a386-4f6d-91ca-9367219465fb" (UID: "8411ae49-a386-4f6d-91ca-9367219465fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:46:47.357278 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.356837    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8411ae49-a386-4f6d-91ca-9367219465fb" (UID: "8411ae49-a386-4f6d-91ca-9367219465fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:46:47.357278 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.357050    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8411ae49-a386-4f6d-91ca-9367219465fb" (UID: "8411ae49-a386-4f6d-91ca-9367219465fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:46:47.357278 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.357239    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-istio-data" (OuterVolumeSpecName: "istio-data") pod "8411ae49-a386-4f6d-91ca-9367219465fb" (UID: "8411ae49-a386-4f6d-91ca-9367219465fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:46:47.357756 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.357657    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8411ae49-a386-4f6d-91ca-9367219465fb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8411ae49-a386-4f6d-91ca-9367219465fb" (UID: "8411ae49-a386-4f6d-91ca-9367219465fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:46:47.359617 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.359585    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8411ae49-a386-4f6d-91ca-9367219465fb-istio-token" (OuterVolumeSpecName: "istio-token") pod "8411ae49-a386-4f6d-91ca-9367219465fb" (UID: "8411ae49-a386-4f6d-91ca-9367219465fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:46:47.359894 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.359860    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8411ae49-a386-4f6d-91ca-9367219465fb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8411ae49-a386-4f6d-91ca-9367219465fb" (UID: "8411ae49-a386-4f6d-91ca-9367219465fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:46:47.360524 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.360500    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8411ae49-a386-4f6d-91ca-9367219465fb" (UID: "8411ae49-a386-4f6d-91ca-9367219465fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:46:47.360892 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.360869    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8411ae49-a386-4f6d-91ca-9367219465fb-kube-api-access-z2ghm" (OuterVolumeSpecName: "kube-api-access-z2ghm") pod "8411ae49-a386-4f6d-91ca-9367219465fb" (UID: "8411ae49-a386-4f6d-91ca-9367219465fb"). InnerVolumeSpecName "kube-api-access-z2ghm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:46:47.368788 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.368765    2578 generic.go:356] "Generic (PLEG): container finished" podID="8411ae49-a386-4f6d-91ca-9367219465fb" containerID="b4f6513c551aaf9abff1452c5a79c17e0aee01879f4b7fcd960c76fa3fc7ae8a" exitCode=0
Sep 09 21:46:47.368887 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.368830    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"
Sep 09 21:46:47.368887 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.368852    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg" event={"ID":"8411ae49-a386-4f6d-91ca-9367219465fb","Type":"ContainerDied","Data":"b4f6513c551aaf9abff1452c5a79c17e0aee01879f4b7fcd960c76fa3fc7ae8a"}
Sep 09 21:46:47.369048 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.368912    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg" event={"ID":"8411ae49-a386-4f6d-91ca-9367219465fb","Type":"ContainerDied","Data":"38a0e2bd15a459f77b48fcc1b4e6d44f38d458704096943283551230a301b5b1"}
Sep 09 21:46:47.369048 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.368948    2578 scope.go:117] "RemoveContainer" containerID="b4f6513c551aaf9abff1452c5a79c17e0aee01879f4b7fcd960c76fa3fc7ae8a"
Sep 09 21:46:47.379056 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.379039    2578 scope.go:117] "RemoveContainer" containerID="120b9e4285dd2f9fc4628a944b0e2843ac604500cb8de7f7127081af122e578a"
Sep 09 21:46:47.387185 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.387171    2578 scope.go:117] "RemoveContainer" containerID="b4f6513c551aaf9abff1452c5a79c17e0aee01879f4b7fcd960c76fa3fc7ae8a"
Sep 09 21:46:47.387435 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:46:47.387420    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4f6513c551aaf9abff1452c5a79c17e0aee01879f4b7fcd960c76fa3fc7ae8a\": container with ID starting with b4f6513c551aaf9abff1452c5a79c17e0aee01879f4b7fcd960c76fa3fc7ae8a not found: ID does not exist" containerID="b4f6513c551aaf9abff1452c5a79c17e0aee01879f4b7fcd960c76fa3fc7ae8a"
Sep 09 21:46:47.387475 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.387442    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f6513c551aaf9abff1452c5a79c17e0aee01879f4b7fcd960c76fa3fc7ae8a"} err="failed to get container status \"b4f6513c551aaf9abff1452c5a79c17e0aee01879f4b7fcd960c76fa3fc7ae8a\": rpc error: code = NotFound desc = could not find container \"b4f6513c551aaf9abff1452c5a79c17e0aee01879f4b7fcd960c76fa3fc7ae8a\": container with ID starting with b4f6513c551aaf9abff1452c5a79c17e0aee01879f4b7fcd960c76fa3fc7ae8a not found: ID does not exist"
Sep 09 21:46:47.387475 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.387456    2578 scope.go:117] "RemoveContainer" containerID="120b9e4285dd2f9fc4628a944b0e2843ac604500cb8de7f7127081af122e578a"
Sep 09 21:46:47.387687 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:46:47.387669    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"120b9e4285dd2f9fc4628a944b0e2843ac604500cb8de7f7127081af122e578a\": container with ID starting with 120b9e4285dd2f9fc4628a944b0e2843ac604500cb8de7f7127081af122e578a not found: ID does not exist" containerID="120b9e4285dd2f9fc4628a944b0e2843ac604500cb8de7f7127081af122e578a"
Sep 09 21:46:47.387744 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.387695    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"120b9e4285dd2f9fc4628a944b0e2843ac604500cb8de7f7127081af122e578a"} err="failed to get container status \"120b9e4285dd2f9fc4628a944b0e2843ac604500cb8de7f7127081af122e578a\": rpc error: code = NotFound desc = could not find container \"120b9e4285dd2f9fc4628a944b0e2843ac604500cb8de7f7127081af122e578a\": container with ID starting with 120b9e4285dd2f9fc4628a944b0e2843ac604500cb8de7f7127081af122e578a not found: ID does not exist"
Sep 09 21:46:47.391422 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.391402    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"]
Sep 09 21:46:47.396765 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.396743    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-04d63732-openshift-default-5455bb5f4b-gkmgg"]
Sep 09 21:46:47.456911 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.456836    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:46:47.456911 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.456871    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8411ae49-a386-4f6d-91ca-9367219465fb-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:46:47.456911 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.456884    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:46:47.457191 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.456898    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8411ae49-a386-4f6d-91ca-9367219465fb-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:46:47.457191 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.457023    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:46:47.457191 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.457038    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2ghm\" (UniqueName: \"kubernetes.io/projected/8411ae49-a386-4f6d-91ca-9367219465fb-kube-api-access-z2ghm\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:46:47.457191 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.457050    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8411ae49-a386-4f6d-91ca-9367219465fb-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:46:47.457191 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.457064    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:46:47.457191 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.457076    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8411ae49-a386-4f6d-91ca-9367219465fb-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:46:47.667098 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.667058    2578 patch_prober.go:28] interesting pod/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 09 21:46:47.667516 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:47.667133    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5" podUID="e5ca4200-2da3-459c-9feb-deeb311d1d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 09 21:46:48.667263 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:48.667226    2578 patch_prober.go:28] interesting pod/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 09 21:46:48.667650 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:48.667290    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5" podUID="e5ca4200-2da3-459c-9feb-deeb311d1d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 09 21:46:49.003838 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:49.003758    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8411ae49-a386-4f6d-91ca-9367219465fb" path="/var/lib/kubelet/pods/8411ae49-a386-4f6d-91ca-9367219465fb/volumes"
Sep 09 21:46:49.666701 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:49.666668    2578 patch_prober.go:28] interesting pod/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 09 21:46:49.666883 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:49.666723    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5" podUID="e5ca4200-2da3-459c-9feb-deeb311d1d70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 09 21:46:50.670785 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:50.670754    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:50.671174 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:50.671074    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:50.671704 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:50.671685    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:46:51.370546 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:46:51.370508    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:47:01.395105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:01.395070    2578 patch_prober.go:28] interesting pod/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 21:47:01.396410 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:01.395151    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t" podUID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 21:47:02.459930 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:02.459890    2578 generic.go:356] "Generic (PLEG): container finished" podID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" containerID="41b1cedaca09fe2d67e982a307575f992c169a9bd165bab221b95e67b5495781" exitCode=0
Sep 09 21:47:02.460314 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:02.459985    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t" event={"ID":"5eb2f107-6cb3-47f6-8bbf-14ce82818c19","Type":"ContainerDied","Data":"41b1cedaca09fe2d67e982a307575f992c169a9bd165bab221b95e67b5495781"}
Sep 09 21:47:02.460448 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:02.460431    2578 scope.go:117] "RemoveContainer" containerID="41b1cedaca09fe2d67e982a307575f992c169a9bd165bab221b95e67b5495781"
Sep 09 21:47:02.461576 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:02.461549    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:47:02.461631 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:02.461617    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:47:02.461667 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:02.461649    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:47:03.468441 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:03.468408    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t" event={"ID":"5eb2f107-6cb3-47f6-8bbf-14ce82818c19","Type":"ContainerStarted","Data":"3026e4c7295f2c360cac9a83684803481756a77dfeaf99beaebd712b0a77dea7"}
Sep 09 21:47:04.159186 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:04.159148    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:47:04.159374 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:04.159346    2578 patch_prober.go:28] interesting pod/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 09 21:47:04.159434 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:04.159388    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t" podUID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 09 21:47:05.159757 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:05.159707    2578 patch_prober.go:28] interesting pod/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 09 21:47:05.160208 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:05.159787    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t" podUID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 09 21:47:06.159774 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:06.159739    2578 patch_prober.go:28] interesting pod/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 09 21:47:06.160192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:06.159809    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t" podUID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 09 21:47:07.159739 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:07.159695    2578 patch_prober.go:28] interesting pod/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 09 21:47:07.159918 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:07.159779    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t" podUID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 09 21:47:08.160437 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:08.160395    2578 patch_prober.go:28] interesting pod/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 09 21:47:08.160913 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:08.160466    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t" podUID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 09 21:47:09.163363 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:09.163334    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:47:09.163772 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:09.163755    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:47:09.164495 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:09.164476    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:47:10.274453 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:10.274419    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:47:17.921945 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:17.921907    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"]
Sep 09 21:47:17.922461 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:17.922147    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t" podUID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" containerName="istio-proxy" containerID="cri-o://3026e4c7295f2c360cac9a83684803481756a77dfeaf99beaebd712b0a77dea7" gracePeriod=30
Sep 09 21:47:18.724560 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.724526    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"]
Sep 09 21:47:18.724945 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.724932    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8411ae49-a386-4f6d-91ca-9367219465fb" containerName="istio-proxy"
Sep 09 21:47:18.725027 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.724946    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8411ae49-a386-4f6d-91ca-9367219465fb" containerName="istio-proxy"
Sep 09 21:47:18.725027 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.724959    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8411ae49-a386-4f6d-91ca-9367219465fb" containerName="istio-proxy"
Sep 09 21:47:18.725027 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.724979    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8411ae49-a386-4f6d-91ca-9367219465fb" containerName="istio-proxy"
Sep 09 21:47:18.725136 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.725079    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8411ae49-a386-4f6d-91ca-9367219465fb" containerName="istio-proxy"
Sep 09 21:47:18.725136 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.725089    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8411ae49-a386-4f6d-91ca-9367219465fb" containerName="istio-proxy"
Sep 09 21:47:18.727889 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.727873    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.730399 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.730378    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b27eb4af-openshift-default-dockercfg-27k2r\""
Sep 09 21:47:18.755234 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.755208    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"]
Sep 09 21:47:18.875650 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.875613    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-workload-certs\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.875650 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.875651    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tndqp\" (UniqueName: \"kubernetes.io/projected/467ca3c8-3540-4490-acb8-268e177a5c19-kube-api-access-tndqp\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.875866 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.875678    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-credential-socket\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.875866 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.875699    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/467ca3c8-3540-4490-acb8-268e177a5c19-istio-podinfo\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.875866 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.875772    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/467ca3c8-3540-4490-acb8-268e177a5c19-istiod-ca-cert\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.875866 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.875813    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-istio-data\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.876038 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.875871    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-istio-envoy\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.876038 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.875898    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/467ca3c8-3540-4490-acb8-268e177a5c19-istio-token\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.876038 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.876002    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-workload-socket\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.976878 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.976789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-workload-certs\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.976878 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.976828    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tndqp\" (UniqueName: \"kubernetes.io/projected/467ca3c8-3540-4490-acb8-268e177a5c19-kube-api-access-tndqp\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.976878 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.976854    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-credential-socket\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.976878 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.976871    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/467ca3c8-3540-4490-acb8-268e177a5c19-istio-podinfo\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.977583 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.977020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/467ca3c8-3540-4490-acb8-268e177a5c19-istiod-ca-cert\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.977583 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.977081    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-istio-data\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.977583 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.977122    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-istio-envoy\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.977583 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.977144    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/467ca3c8-3540-4490-acb8-268e177a5c19-istio-token\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.977583 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.977262    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-workload-certs\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.977583 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.977269    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-workload-socket\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.977583 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.977303    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-credential-socket\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.977858 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.977618    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-workload-socket\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.977858 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.977715    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-istio-data\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.977858 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.977800    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/467ca3c8-3540-4490-acb8-268e177a5c19-istiod-ca-cert\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.979456 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.979431    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-istio-envoy\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.979783 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.979761    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/467ca3c8-3540-4490-acb8-268e177a5c19-istio-podinfo\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.991261 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.991231    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/467ca3c8-3540-4490-acb8-268e177a5c19-istio-token\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:18.991639 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:18.991622    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tndqp\" (UniqueName: \"kubernetes.io/projected/467ca3c8-3540-4490-acb8-268e177a5c19-kube-api-access-tndqp\") pod \"e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") " pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:19.039683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:19.039659    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:19.317483 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:19.317446    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"]
Sep 09 21:47:19.318417 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:47:19.318394    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod467ca3c8_3540_4490_acb8_268e177a5c19.slice/crio-2239874580e61b6cef5466457c474721a7f7d013e72b265980251752dd7185b0 WatchSource:0}: Error finding container 2239874580e61b6cef5466457c474721a7f7d013e72b265980251752dd7185b0: Status 404 returned error can't find the container with id 2239874580e61b6cef5466457c474721a7f7d013e72b265980251752dd7185b0
Sep 09 21:47:19.320739 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:19.320707    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:47:19.320817 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:19.320769    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:47:19.320817 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:19.320799    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:47:19.583589 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:19.583497    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj" event={"ID":"467ca3c8-3540-4490-acb8-268e177a5c19","Type":"ContainerStarted","Data":"f5b4459db329be91ecc72873cb7a0cfed3bb6f8ed3db06d54b9012579dd55912"}
Sep 09 21:47:19.583732 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:19.583607    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj" event={"ID":"467ca3c8-3540-4490-acb8-268e177a5c19","Type":"ContainerStarted","Data":"2239874580e61b6cef5466457c474721a7f7d013e72b265980251752dd7185b0"}
Sep 09 21:47:19.610095 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:19.610028    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj" podStartSLOduration=1.6100147900000001 podStartE2EDuration="1.61001479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:47:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:47:19.608264169 +0000 UTC m=+859.081229079" watchObservedRunningTime="2026-09-09 21:47:19.61001479 +0000 UTC m=+859.082979700"
Sep 09 21:47:20.040040 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:20.040009    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:20.045105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:20.045083    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:20.589104 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:20.589069    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:20.590118 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:20.590096    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:23.217866 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.217838    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:47:23.316456 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.316435    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zt5c7\" (UniqueName: \"kubernetes.io/projected/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-kube-api-access-zt5c7\") pod \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") "
Sep 09 21:47:23.316590 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.316466    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-token\") pod \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") "
Sep 09 21:47:23.316590 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.316490    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istiod-ca-cert\") pod \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") "
Sep 09 21:47:23.316590 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.316520    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-podinfo\") pod \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") "
Sep 09 21:47:23.316590 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.316551    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-workload-socket\") pod \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") "
Sep 09 21:47:23.316590 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.316585    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-data\") pod \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") "
Sep 09 21:47:23.316831 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.316626    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-envoy\") pod \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") "
Sep 09 21:47:23.316831 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.316659    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-credential-socket\") pod \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") "
Sep 09 21:47:23.316831 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.316726    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-workload-certs\") pod \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\" (UID: \"5eb2f107-6cb3-47f6-8bbf-14ce82818c19\") "
Sep 09 21:47:23.317003 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.316885    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5eb2f107-6cb3-47f6-8bbf-14ce82818c19" (UID: "5eb2f107-6cb3-47f6-8bbf-14ce82818c19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:47:23.317003 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.316934    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5eb2f107-6cb3-47f6-8bbf-14ce82818c19" (UID: "5eb2f107-6cb3-47f6-8bbf-14ce82818c19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:47:23.317003 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.316942    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-data" (OuterVolumeSpecName: "istio-data") pod "5eb2f107-6cb3-47f6-8bbf-14ce82818c19" (UID: "5eb2f107-6cb3-47f6-8bbf-14ce82818c19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:47:23.317176 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.317074    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5eb2f107-6cb3-47f6-8bbf-14ce82818c19" (UID: "5eb2f107-6cb3-47f6-8bbf-14ce82818c19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:47:23.317176 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.317128    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:47:23.317176 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.317144    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:47:23.317176 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.317158    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:47:23.317351 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.317180    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5eb2f107-6cb3-47f6-8bbf-14ce82818c19" (UID: "5eb2f107-6cb3-47f6-8bbf-14ce82818c19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:47:23.318807 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.318780    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5eb2f107-6cb3-47f6-8bbf-14ce82818c19" (UID: "5eb2f107-6cb3-47f6-8bbf-14ce82818c19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:47:23.318901 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.318861    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-token" (OuterVolumeSpecName: "istio-token") pod "5eb2f107-6cb3-47f6-8bbf-14ce82818c19" (UID: "5eb2f107-6cb3-47f6-8bbf-14ce82818c19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:47:23.319132 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.319115    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5eb2f107-6cb3-47f6-8bbf-14ce82818c19" (UID: "5eb2f107-6cb3-47f6-8bbf-14ce82818c19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:47:23.319189 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.319128    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-kube-api-access-zt5c7" (OuterVolumeSpecName: "kube-api-access-zt5c7") pod "5eb2f107-6cb3-47f6-8bbf-14ce82818c19" (UID: "5eb2f107-6cb3-47f6-8bbf-14ce82818c19"). InnerVolumeSpecName "kube-api-access-zt5c7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:47:23.418466 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.418443    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:47:23.418575 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.418470    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:47:23.418575 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.418484    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt5c7\" (UniqueName: \"kubernetes.io/projected/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-kube-api-access-zt5c7\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:47:23.418575 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.418499    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:47:23.418575 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.418511    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:47:23.418575 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.418522    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eb2f107-6cb3-47f6-8bbf-14ce82818c19-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:47:23.607758 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.607729    2578 generic.go:356] "Generic (PLEG): container finished" podID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" containerID="3026e4c7295f2c360cac9a83684803481756a77dfeaf99beaebd712b0a77dea7" exitCode=0
Sep 09 21:47:23.607871 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.607770    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"
Sep 09 21:47:23.607871 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.607784    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t" event={"ID":"5eb2f107-6cb3-47f6-8bbf-14ce82818c19","Type":"ContainerDied","Data":"3026e4c7295f2c360cac9a83684803481756a77dfeaf99beaebd712b0a77dea7"}
Sep 09 21:47:23.607871 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.607838    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t" event={"ID":"5eb2f107-6cb3-47f6-8bbf-14ce82818c19","Type":"ContainerDied","Data":"e6c403c15e3c773b797f7794e8138bdad2f269ae076bf2ba7ac7d46295127914"}
Sep 09 21:47:23.607871 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.607861    2578 scope.go:117] "RemoveContainer" containerID="3026e4c7295f2c360cac9a83684803481756a77dfeaf99beaebd712b0a77dea7"
Sep 09 21:47:23.617962 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.617939    2578 scope.go:117] "RemoveContainer" containerID="41b1cedaca09fe2d67e982a307575f992c169a9bd165bab221b95e67b5495781"
Sep 09 21:47:23.629242 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.629221    2578 scope.go:117] "RemoveContainer" containerID="3026e4c7295f2c360cac9a83684803481756a77dfeaf99beaebd712b0a77dea7"
Sep 09 21:47:23.629556 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:47:23.629535    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3026e4c7295f2c360cac9a83684803481756a77dfeaf99beaebd712b0a77dea7\": container with ID starting with 3026e4c7295f2c360cac9a83684803481756a77dfeaf99beaebd712b0a77dea7 not found: ID does not exist" containerID="3026e4c7295f2c360cac9a83684803481756a77dfeaf99beaebd712b0a77dea7"
Sep 09 21:47:23.629603 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.629565    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3026e4c7295f2c360cac9a83684803481756a77dfeaf99beaebd712b0a77dea7"} err="failed to get container status \"3026e4c7295f2c360cac9a83684803481756a77dfeaf99beaebd712b0a77dea7\": rpc error: code = NotFound desc = could not find container \"3026e4c7295f2c360cac9a83684803481756a77dfeaf99beaebd712b0a77dea7\": container with ID starting with 3026e4c7295f2c360cac9a83684803481756a77dfeaf99beaebd712b0a77dea7 not found: ID does not exist"
Sep 09 21:47:23.629603 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.629584    2578 scope.go:117] "RemoveContainer" containerID="41b1cedaca09fe2d67e982a307575f992c169a9bd165bab221b95e67b5495781"
Sep 09 21:47:23.629895 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:47:23.629880    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41b1cedaca09fe2d67e982a307575f992c169a9bd165bab221b95e67b5495781\": container with ID starting with 41b1cedaca09fe2d67e982a307575f992c169a9bd165bab221b95e67b5495781 not found: ID does not exist" containerID="41b1cedaca09fe2d67e982a307575f992c169a9bd165bab221b95e67b5495781"
Sep 09 21:47:23.629937 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.629898    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41b1cedaca09fe2d67e982a307575f992c169a9bd165bab221b95e67b5495781"} err="failed to get container status \"41b1cedaca09fe2d67e982a307575f992c169a9bd165bab221b95e67b5495781\": rpc error: code = NotFound desc = could not find container \"41b1cedaca09fe2d67e982a307575f992c169a9bd165bab221b95e67b5495781\": container with ID starting with 41b1cedaca09fe2d67e982a307575f992c169a9bd165bab221b95e67b5495781 not found: ID does not exist"
Sep 09 21:47:23.631934 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.631913    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"]
Sep 09 21:47:23.639048 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:23.639015    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-98c7fb47-gw-openshift-default-7d98597448-q876t"]
Sep 09 21:47:24.875984 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:24.875928    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:47:25.004215 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:25.004182    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" path="/var/lib/kubelet/pods/5eb2f107-6cb3-47f6-8bbf-14ce82818c19/volumes"
Sep 09 21:47:39.982183 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:39.982144    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:47:43.744507 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:43.744481    2578 generic.go:356] "Generic (PLEG): container finished" podID="467ca3c8-3540-4490-acb8-268e177a5c19" containerID="f5b4459db329be91ecc72873cb7a0cfed3bb6f8ed3db06d54b9012579dd55912" exitCode=0
Sep 09 21:47:43.744822 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:43.744556    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj" event={"ID":"467ca3c8-3540-4490-acb8-268e177a5c19","Type":"ContainerDied","Data":"f5b4459db329be91ecc72873cb7a0cfed3bb6f8ed3db06d54b9012579dd55912"}
Sep 09 21:47:43.745040 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:43.745019    2578 scope.go:117] "RemoveContainer" containerID="f5b4459db329be91ecc72873cb7a0cfed3bb6f8ed3db06d54b9012579dd55912"
Sep 09 21:47:43.746199 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:43.746169    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:47:43.746307 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:43.746252    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:47:43.746372 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:43.746306    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:47:44.758041 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:44.758003    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj" event={"ID":"467ca3c8-3540-4490-acb8-268e177a5c19","Type":"ContainerStarted","Data":"5c645fa222c0b080f154715d30df5582a1546f314083f6701c2494549d708806"}
Sep 09 21:47:45.040590 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:45.040555    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:45.040948 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:45.040921    2578 patch_prober.go:28] interesting pod/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 09 21:47:45.041081 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:45.040997    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj" podUID="467ca3c8-3540-4490-acb8-268e177a5c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 09 21:47:46.040960 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:46.040921    2578 patch_prober.go:28] interesting pod/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 09 21:47:46.041560 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:46.041528    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj" podUID="467ca3c8-3540-4490-acb8-268e177a5c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 09 21:47:47.040902 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:47.040868    2578 patch_prober.go:28] interesting pod/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 09 21:47:47.041222 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:47.041193    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj" podUID="467ca3c8-3540-4490-acb8-268e177a5c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 09 21:47:48.040430 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:48.040390    2578 patch_prober.go:28] interesting pod/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 09 21:47:48.040728 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:48.040700    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj" podUID="467ca3c8-3540-4490-acb8-268e177a5c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 09 21:47:49.040269 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:49.040239    2578 patch_prober.go:28] interesting pod/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 09 21:47:49.040947 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:49.040918    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj" podUID="467ca3c8-3540-4490-acb8-268e177a5c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 09 21:47:50.040948 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:50.040908    2578 patch_prober.go:28] interesting pod/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 09 21:47:50.041373 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:50.041005    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj" podUID="467ca3c8-3540-4490-acb8-268e177a5c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 09 21:47:50.589585 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:50.589557    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:51.044186 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:51.044158    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:51.045002 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:51.044979    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:47:51.124134 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:51.124095    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:47:55.292489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:55.292416    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"]
Sep 09 21:47:55.292900 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:55.292628    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj" podUID="467ca3c8-3540-4490-acb8-268e177a5c19" containerName="istio-proxy" containerID="cri-o://5c645fa222c0b080f154715d30df5582a1546f314083f6701c2494549d708806" gracePeriod=30
Sep 09 21:47:56.110510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.110475    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"]
Sep 09 21:47:56.110888 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.110876    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" containerName="istio-proxy"
Sep 09 21:47:56.110934 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.110889    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" containerName="istio-proxy"
Sep 09 21:47:56.111001 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.110985    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" containerName="istio-proxy"
Sep 09 21:47:56.111001 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.110992    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" containerName="istio-proxy"
Sep 09 21:47:56.113785 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.113771    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.117018 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.116998    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-ef9b2234-openshift-default-dockercfg-4f8tt\""
Sep 09 21:47:56.140580 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.140549    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"]
Sep 09 21:47:56.238220 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.238186    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxbkt\" (UniqueName: \"kubernetes.io/projected/97fd8ab3-c645-439c-a4c6-179778662cf5-kube-api-access-cxbkt\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.238392 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.238222    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-data\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.238392 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.238250    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-envoy\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.238392 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.238334    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-credential-socket\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.238392 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.238374    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-workload-certs\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.238560 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.238410    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-workload-socket\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.238560 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.238455    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-token\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.238560 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.238549    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97fd8ab3-c645-439c-a4c6-179778662cf5-istiod-ca-cert\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.238664 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.238599    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-podinfo\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.339443 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.339412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97fd8ab3-c645-439c-a4c6-179778662cf5-istiod-ca-cert\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.339875 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.339460    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-podinfo\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.339875 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.339479    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxbkt\" (UniqueName: \"kubernetes.io/projected/97fd8ab3-c645-439c-a4c6-179778662cf5-kube-api-access-cxbkt\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.339875 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.339498    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-data\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.339875 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.339517    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-envoy\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.339875 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.339545    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-credential-socket\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.339875 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.339574    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-workload-certs\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.339875 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.339726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-workload-socket\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.339875 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.339809    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-token\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.340338 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.340178    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97fd8ab3-c645-439c-a4c6-179778662cf5-istiod-ca-cert\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.340519 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.340489    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-credential-socket\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.340642 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.340567    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-workload-certs\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.340642 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.340594    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-workload-socket\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.340642 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.340618    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-data\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.342038 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.342019    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-podinfo\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.342175 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.342159    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-envoy\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.358703 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.358678    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-token\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.358916 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.358893    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxbkt\" (UniqueName: \"kubernetes.io/projected/97fd8ab3-c645-439c-a4c6-179778662cf5-kube-api-access-cxbkt\") pod \"e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") " pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.424263 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.424214    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:56.726310 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.726287    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"]
Sep 09 21:47:56.727524 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:47:56.727496    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97fd8ab3_c645_439c_a4c6_179778662cf5.slice/crio-7b08ced8407356c82f095a456ac98de5d8d9b3e071a98359b5a42e8003af127c WatchSource:0}: Error finding container 7b08ced8407356c82f095a456ac98de5d8d9b3e071a98359b5a42e8003af127c: Status 404 returned error can't find the container with id 7b08ced8407356c82f095a456ac98de5d8d9b3e071a98359b5a42e8003af127c
Sep 09 21:47:56.729816 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.729784    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:47:56.729905 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.729846    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:47:56.729905 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.729876    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:47:56.834807 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.834768    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d" event={"ID":"97fd8ab3-c645-439c-a4c6-179778662cf5","Type":"ContainerStarted","Data":"cb1ba6b37715461beeec7076e33f1f56c800582d355e8c061334ba7539134f32"}
Sep 09 21:47:56.834992 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.834870    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d" event={"ID":"97fd8ab3-c645-439c-a4c6-179778662cf5","Type":"ContainerStarted","Data":"7b08ced8407356c82f095a456ac98de5d8d9b3e071a98359b5a42e8003af127c"}
Sep 09 21:47:56.882606 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:56.882556    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d" podStartSLOduration=0.882539192 podStartE2EDuration="882.539192ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:47:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:47:56.881396489 +0000 UTC m=+896.354361400" watchObservedRunningTime="2026-09-09 21:47:56.882539192 +0000 UTC m=+896.355504101"
Sep 09 21:47:57.424395 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:57.424360    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:57.429311 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:57.429288    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:57.841104 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:57.841074    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:47:57.842083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:47:57.842061    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:48:00.585264 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.585244    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:48:00.704053 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704016    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-istio-envoy\") pod \"467ca3c8-3540-4490-acb8-268e177a5c19\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") "
Sep 09 21:48:00.704053 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704057    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/467ca3c8-3540-4490-acb8-268e177a5c19-istiod-ca-cert\") pod \"467ca3c8-3540-4490-acb8-268e177a5c19\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") "
Sep 09 21:48:00.704280 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704111    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-istio-data\") pod \"467ca3c8-3540-4490-acb8-268e177a5c19\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") "
Sep 09 21:48:00.704280 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704144    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-credential-socket\") pod \"467ca3c8-3540-4490-acb8-268e177a5c19\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") "
Sep 09 21:48:00.704280 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704183    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/467ca3c8-3540-4490-acb8-268e177a5c19-istio-token\") pod \"467ca3c8-3540-4490-acb8-268e177a5c19\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") "
Sep 09 21:48:00.704280 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704262    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tndqp\" (UniqueName: \"kubernetes.io/projected/467ca3c8-3540-4490-acb8-268e177a5c19-kube-api-access-tndqp\") pod \"467ca3c8-3540-4490-acb8-268e177a5c19\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") "
Sep 09 21:48:00.704498 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704342    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-workload-certs\") pod \"467ca3c8-3540-4490-acb8-268e177a5c19\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") "
Sep 09 21:48:00.704498 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704372    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/467ca3c8-3540-4490-acb8-268e177a5c19-istio-podinfo\") pod \"467ca3c8-3540-4490-acb8-268e177a5c19\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") "
Sep 09 21:48:00.704498 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704425    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-workload-socket\") pod \"467ca3c8-3540-4490-acb8-268e177a5c19\" (UID: \"467ca3c8-3540-4490-acb8-268e177a5c19\") "
Sep 09 21:48:00.704498 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704431    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-istio-data" (OuterVolumeSpecName: "istio-data") pod "467ca3c8-3540-4490-acb8-268e177a5c19" (UID: "467ca3c8-3540-4490-acb8-268e177a5c19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:48:00.704498 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704482    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "467ca3c8-3540-4490-acb8-268e177a5c19" (UID: "467ca3c8-3540-4490-acb8-268e177a5c19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:48:00.704753 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704599    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "467ca3c8-3540-4490-acb8-268e177a5c19" (UID: "467ca3c8-3540-4490-acb8-268e177a5c19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:48:00.704753 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704617    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/467ca3c8-3540-4490-acb8-268e177a5c19-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "467ca3c8-3540-4490-acb8-268e177a5c19" (UID: "467ca3c8-3540-4490-acb8-268e177a5c19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:48:00.704855 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704790    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "467ca3c8-3540-4490-acb8-268e177a5c19" (UID: "467ca3c8-3540-4490-acb8-268e177a5c19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:48:00.704907 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704866    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:00.704907 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704887    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/467ca3c8-3540-4490-acb8-268e177a5c19-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:00.704907 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704897    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:00.705070 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704909    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:00.705070 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.704922    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:00.706681 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.706657    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "467ca3c8-3540-4490-acb8-268e177a5c19" (UID: "467ca3c8-3540-4490-acb8-268e177a5c19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:48:00.706919 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.706899    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/467ca3c8-3540-4490-acb8-268e177a5c19-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "467ca3c8-3540-4490-acb8-268e177a5c19" (UID: "467ca3c8-3540-4490-acb8-268e177a5c19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:48:00.707061 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.707044    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/467ca3c8-3540-4490-acb8-268e177a5c19-kube-api-access-tndqp" (OuterVolumeSpecName: "kube-api-access-tndqp") pod "467ca3c8-3540-4490-acb8-268e177a5c19" (UID: "467ca3c8-3540-4490-acb8-268e177a5c19"). InnerVolumeSpecName "kube-api-access-tndqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:48:00.707106 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.707070    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/467ca3c8-3540-4490-acb8-268e177a5c19-istio-token" (OuterVolumeSpecName: "istio-token") pod "467ca3c8-3540-4490-acb8-268e177a5c19" (UID: "467ca3c8-3540-4490-acb8-268e177a5c19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:48:00.805809 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.805781    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tndqp\" (UniqueName: \"kubernetes.io/projected/467ca3c8-3540-4490-acb8-268e177a5c19-kube-api-access-tndqp\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:00.805809 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.805804    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/467ca3c8-3540-4490-acb8-268e177a5c19-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:00.805809 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.805814    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/467ca3c8-3540-4490-acb8-268e177a5c19-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:00.806069 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.805823    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/467ca3c8-3540-4490-acb8-268e177a5c19-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:00.859857 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.859792    2578 generic.go:356] "Generic (PLEG): container finished" podID="467ca3c8-3540-4490-acb8-268e177a5c19" containerID="5c645fa222c0b080f154715d30df5582a1546f314083f6701c2494549d708806" exitCode=0
Sep 09 21:48:00.859857 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.859844    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"
Sep 09 21:48:00.860017 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.859862    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj" event={"ID":"467ca3c8-3540-4490-acb8-268e177a5c19","Type":"ContainerDied","Data":"5c645fa222c0b080f154715d30df5582a1546f314083f6701c2494549d708806"}
Sep 09 21:48:00.860017 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.859917    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj" event={"ID":"467ca3c8-3540-4490-acb8-268e177a5c19","Type":"ContainerDied","Data":"2239874580e61b6cef5466457c474721a7f7d013e72b265980251752dd7185b0"}
Sep 09 21:48:00.860017 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.859945    2578 scope.go:117] "RemoveContainer" containerID="5c645fa222c0b080f154715d30df5582a1546f314083f6701c2494549d708806"
Sep 09 21:48:00.870234 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.870216    2578 scope.go:117] "RemoveContainer" containerID="f5b4459db329be91ecc72873cb7a0cfed3bb6f8ed3db06d54b9012579dd55912"
Sep 09 21:48:00.878575 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.878556    2578 scope.go:117] "RemoveContainer" containerID="5c645fa222c0b080f154715d30df5582a1546f314083f6701c2494549d708806"
Sep 09 21:48:00.878826 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:48:00.878809    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c645fa222c0b080f154715d30df5582a1546f314083f6701c2494549d708806\": container with ID starting with 5c645fa222c0b080f154715d30df5582a1546f314083f6701c2494549d708806 not found: ID does not exist" containerID="5c645fa222c0b080f154715d30df5582a1546f314083f6701c2494549d708806"
Sep 09 21:48:00.878882 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.878836    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c645fa222c0b080f154715d30df5582a1546f314083f6701c2494549d708806"} err="failed to get container status \"5c645fa222c0b080f154715d30df5582a1546f314083f6701c2494549d708806\": rpc error: code = NotFound desc = could not find container \"5c645fa222c0b080f154715d30df5582a1546f314083f6701c2494549d708806\": container with ID starting with 5c645fa222c0b080f154715d30df5582a1546f314083f6701c2494549d708806 not found: ID does not exist"
Sep 09 21:48:00.878882 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.878853    2578 scope.go:117] "RemoveContainer" containerID="f5b4459db329be91ecc72873cb7a0cfed3bb6f8ed3db06d54b9012579dd55912"
Sep 09 21:48:00.879112 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:48:00.879096    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5b4459db329be91ecc72873cb7a0cfed3bb6f8ed3db06d54b9012579dd55912\": container with ID starting with f5b4459db329be91ecc72873cb7a0cfed3bb6f8ed3db06d54b9012579dd55912 not found: ID does not exist" containerID="f5b4459db329be91ecc72873cb7a0cfed3bb6f8ed3db06d54b9012579dd55912"
Sep 09 21:48:00.879162 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.879117    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5b4459db329be91ecc72873cb7a0cfed3bb6f8ed3db06d54b9012579dd55912"} err="failed to get container status \"f5b4459db329be91ecc72873cb7a0cfed3bb6f8ed3db06d54b9012579dd55912\": rpc error: code = NotFound desc = could not find container \"f5b4459db329be91ecc72873cb7a0cfed3bb6f8ed3db06d54b9012579dd55912\": container with ID starting with f5b4459db329be91ecc72873cb7a0cfed3bb6f8ed3db06d54b9012579dd55912 not found: ID does not exist"
Sep 09 21:48:00.883420 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.883399    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"]
Sep 09 21:48:00.888057 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:00.888038    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b27eb4af-openshift-default-978c8d498-sh9vj"]
Sep 09 21:48:01.005081 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:01.005047    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="467ca3c8-3540-4490-acb8-268e177a5c19" path="/var/lib/kubelet/pods/467ca3c8-3540-4490-acb8-268e177a5c19/volumes"
Sep 09 21:48:01.008424 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:01.008403    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 21:48:01.011305 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:01.011285    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 21:48:12.177440 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:12.177406    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:48:23.005065 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:23.005035    2578 generic.go:356] "Generic (PLEG): container finished" podID="97fd8ab3-c645-439c-a4c6-179778662cf5" containerID="cb1ba6b37715461beeec7076e33f1f56c800582d355e8c061334ba7539134f32" exitCode=0
Sep 09 21:48:23.005494 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:23.005114    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d" event={"ID":"97fd8ab3-c645-439c-a4c6-179778662cf5","Type":"ContainerDied","Data":"cb1ba6b37715461beeec7076e33f1f56c800582d355e8c061334ba7539134f32"}
Sep 09 21:48:23.005494 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:23.005458    2578 scope.go:117] "RemoveContainer" containerID="cb1ba6b37715461beeec7076e33f1f56c800582d355e8c061334ba7539134f32"
Sep 09 21:48:23.006562 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:23.006533    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:48:23.006649 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:23.006598    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:48:23.006649 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:23.006628    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:48:24.013960 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:24.013927    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d" event={"ID":"97fd8ab3-c645-439c-a4c6-179778662cf5","Type":"ContainerStarted","Data":"cb376fcf0a0da52e64d39fddee1030bfa2ef3fe15496fbadd3e7b4f51dcfcd8c"}
Sep 09 21:48:24.425350 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:24.425317    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:48:24.426529 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:24.426501    2578 patch_prober.go:28] interesting pod/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 09 21:48:24.426645 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:24.426562    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d" podUID="97fd8ab3-c645-439c-a4c6-179778662cf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 09 21:48:25.237988 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.237940    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-66cedd-r9j7x"]
Sep 09 21:48:25.239226 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.239199    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="467ca3c8-3540-4490-acb8-268e177a5c19" containerName="istio-proxy"
Sep 09 21:48:25.239367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.239356    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="467ca3c8-3540-4490-acb8-268e177a5c19" containerName="istio-proxy"
Sep 09 21:48:25.239456 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.239446    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" containerName="istio-proxy"
Sep 09 21:48:25.239536 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.239527    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eb2f107-6cb3-47f6-8bbf-14ce82818c19" containerName="istio-proxy"
Sep 09 21:48:25.239620 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.239610    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="467ca3c8-3540-4490-acb8-268e177a5c19" containerName="istio-proxy"
Sep 09 21:48:25.239694 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.239685    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="467ca3c8-3540-4490-acb8-268e177a5c19" containerName="istio-proxy"
Sep 09 21:48:25.239864 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.239853    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="467ca3c8-3540-4490-acb8-268e177a5c19" containerName="istio-proxy"
Sep 09 21:48:25.239953 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.239944    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="467ca3c8-3540-4490-acb8-268e177a5c19" containerName="istio-proxy"
Sep 09 21:48:25.242657 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.242639    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-66cedd-r9j7x"
Sep 09 21:48:25.247340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.247312    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65jqq\""
Sep 09 21:48:25.269945 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.269912    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-66cedd-r9j7x"]
Sep 09 21:48:25.354662 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.354629    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd3e8fd6-590a-4303-8b33-c2f369092447-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-66cedd-r9j7x\" (UID: \"fd3e8fd6-590a-4303-8b33-c2f369092447\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-66cedd-r9j7x"
Sep 09 21:48:25.425322 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.425290    2578 patch_prober.go:28] interesting pod/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 09 21:48:25.425470 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.425338    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d" podUID="97fd8ab3-c645-439c-a4c6-179778662cf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 09 21:48:25.456725 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.456689    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd3e8fd6-590a-4303-8b33-c2f369092447-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-66cedd-r9j7x\" (UID: \"fd3e8fd6-590a-4303-8b33-c2f369092447\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-66cedd-r9j7x"
Sep 09 21:48:25.458309 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.458270    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd3e8fd6-590a-4303-8b33-c2f369092447-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-66cedd-r9j7x\" (UID: \"fd3e8fd6-590a-4303-8b33-c2f369092447\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-66cedd-r9j7x"
Sep 09 21:48:25.558255 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.558223    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-66cedd-r9j7x"
Sep 09 21:48:25.836205 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:25.834236    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-66cedd-r9j7x"]
Sep 09 21:48:25.852046 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:48:25.850063    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd3e8fd6_590a_4303_8b33_c2f369092447.slice/crio-5858594b2796e382b88bfa1704b2f6a84cdfb2347dc772ff6bc62393b6c20c28 WatchSource:0}: Error finding container 5858594b2796e382b88bfa1704b2f6a84cdfb2347dc772ff6bc62393b6c20c28: Status 404 returned error can't find the container with id 5858594b2796e382b88bfa1704b2f6a84cdfb2347dc772ff6bc62393b6c20c28
Sep 09 21:48:26.026391 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:26.026363    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-66cedd-r9j7x" event={"ID":"fd3e8fd6-590a-4303-8b33-c2f369092447","Type":"ContainerStarted","Data":"5858594b2796e382b88bfa1704b2f6a84cdfb2347dc772ff6bc62393b6c20c28"}
Sep 09 21:48:26.425166 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:26.425091    2578 patch_prober.go:28] interesting pod/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 09 21:48:26.425166 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:26.425141    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d" podUID="97fd8ab3-c645-439c-a4c6-179778662cf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 09 21:48:27.035645 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:27.035607    2578 generic.go:356] "Generic (PLEG): container finished" podID="fd3e8fd6-590a-4303-8b33-c2f369092447" containerID="bbfbfbd938e08e3e530271e85f78ea4c9dbf682c2be0ec2e091dbabe606c82bf" exitCode=0
Sep 09 21:48:27.035837 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:27.035725    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-66cedd-r9j7x" event={"ID":"fd3e8fd6-590a-4303-8b33-c2f369092447","Type":"ContainerDied","Data":"bbfbfbd938e08e3e530271e85f78ea4c9dbf682c2be0ec2e091dbabe606c82bf"}
Sep 09 21:48:27.425484 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:27.425399    2578 patch_prober.go:28] interesting pod/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 09 21:48:27.425484 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:27.425450    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d" podUID="97fd8ab3-c645-439c-a4c6-179778662cf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 09 21:48:27.842030 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:27.841193    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:48:28.194035 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:28.194013    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-66cedd-r9j7x"
Sep 09 21:48:28.286278 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:28.286252    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd3e8fd6-590a-4303-8b33-c2f369092447-maas-api-service-ca\") pod \"fd3e8fd6-590a-4303-8b33-c2f369092447\" (UID: \"fd3e8fd6-590a-4303-8b33-c2f369092447\") "
Sep 09 21:48:28.286584 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:28.286563    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd3e8fd6-590a-4303-8b33-c2f369092447-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fd3e8fd6-590a-4303-8b33-c2f369092447" (UID: "fd3e8fd6-590a-4303-8b33-c2f369092447"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:48:28.387411 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:28.387331    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd3e8fd6-590a-4303-8b33-c2f369092447-maas-api-service-ca\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:28.424935 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:28.424907    2578 patch_prober.go:28] interesting pod/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 09 21:48:28.425066 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:28.424952    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d" podUID="97fd8ab3-c645-439c-a4c6-179778662cf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 09 21:48:29.051815 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:29.051791    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-66cedd-r9j7x"
Sep 09 21:48:29.052158 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:29.051791    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-66cedd-r9j7x" event={"ID":"fd3e8fd6-590a-4303-8b33-c2f369092447","Type":"ContainerDied","Data":"5858594b2796e382b88bfa1704b2f6a84cdfb2347dc772ff6bc62393b6c20c28"}
Sep 09 21:48:29.052158 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:29.051912    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5858594b2796e382b88bfa1704b2f6a84cdfb2347dc772ff6bc62393b6c20c28"
Sep 09 21:48:29.428515 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:29.428485    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:48:29.429489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:29.429461    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:48:30.931673 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:30.931635    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"]
Sep 09 21:48:31.063553 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:31.063517    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d" podUID="97fd8ab3-c645-439c-a4c6-179778662cf5" containerName="istio-proxy" containerID="cri-o://cb376fcf0a0da52e64d39fddee1030bfa2ef3fe15496fbadd3e7b4f51dcfcd8c" gracePeriod=30
Sep 09 21:48:35.868363 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:35.868330    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:48:36.236550 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.235666    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-66cedd-r9j7x"]
Sep 09 21:48:36.238176 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.238143    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-66cedd-r9j7x"]
Sep 09 21:48:36.299295 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.299276    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:48:36.463191 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.463113    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-workload-socket\") pod \"97fd8ab3-c645-439c-a4c6-179778662cf5\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") "
Sep 09 21:48:36.463191 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.463152    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-data\") pod \"97fd8ab3-c645-439c-a4c6-179778662cf5\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") "
Sep 09 21:48:36.463191 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.463178    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97fd8ab3-c645-439c-a4c6-179778662cf5-istiod-ca-cert\") pod \"97fd8ab3-c645-439c-a4c6-179778662cf5\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") "
Sep 09 21:48:36.463479 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.463202    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-podinfo\") pod \"97fd8ab3-c645-439c-a4c6-179778662cf5\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") "
Sep 09 21:48:36.463479 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.463245    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxbkt\" (UniqueName: \"kubernetes.io/projected/97fd8ab3-c645-439c-a4c6-179778662cf5-kube-api-access-cxbkt\") pod \"97fd8ab3-c645-439c-a4c6-179778662cf5\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") "
Sep 09 21:48:36.463479 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.463276    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-credential-socket\") pod \"97fd8ab3-c645-439c-a4c6-179778662cf5\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") "
Sep 09 21:48:36.463479 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.463316    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-token\") pod \"97fd8ab3-c645-439c-a4c6-179778662cf5\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") "
Sep 09 21:48:36.463479 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.463353    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-envoy\") pod \"97fd8ab3-c645-439c-a4c6-179778662cf5\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") "
Sep 09 21:48:36.463479 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.463380    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-workload-certs\") pod \"97fd8ab3-c645-439c-a4c6-179778662cf5\" (UID: \"97fd8ab3-c645-439c-a4c6-179778662cf5\") "
Sep 09 21:48:36.463784 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.463478    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "97fd8ab3-c645-439c-a4c6-179778662cf5" (UID: "97fd8ab3-c645-439c-a4c6-179778662cf5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:48:36.463784 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.463692    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "97fd8ab3-c645-439c-a4c6-179778662cf5" (UID: "97fd8ab3-c645-439c-a4c6-179778662cf5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:48:36.463784 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.463720    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97fd8ab3-c645-439c-a4c6-179778662cf5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "97fd8ab3-c645-439c-a4c6-179778662cf5" (UID: "97fd8ab3-c645-439c-a4c6-179778662cf5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:48:36.463784 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.463741    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:36.464004 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.463797    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-data" (OuterVolumeSpecName: "istio-data") pod "97fd8ab3-c645-439c-a4c6-179778662cf5" (UID: "97fd8ab3-c645-439c-a4c6-179778662cf5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:48:36.464004 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.463894    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "97fd8ab3-c645-439c-a4c6-179778662cf5" (UID: "97fd8ab3-c645-439c-a4c6-179778662cf5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:48:36.465728 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.465695    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "97fd8ab3-c645-439c-a4c6-179778662cf5" (UID: "97fd8ab3-c645-439c-a4c6-179778662cf5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:48:36.465859 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.465838    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97fd8ab3-c645-439c-a4c6-179778662cf5-kube-api-access-cxbkt" (OuterVolumeSpecName: "kube-api-access-cxbkt") pod "97fd8ab3-c645-439c-a4c6-179778662cf5" (UID: "97fd8ab3-c645-439c-a4c6-179778662cf5"). InnerVolumeSpecName "kube-api-access-cxbkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:48:36.465916 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.465895    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "97fd8ab3-c645-439c-a4c6-179778662cf5" (UID: "97fd8ab3-c645-439c-a4c6-179778662cf5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:48:36.466078 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.466056    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-token" (OuterVolumeSpecName: "istio-token") pod "97fd8ab3-c645-439c-a4c6-179778662cf5" (UID: "97fd8ab3-c645-439c-a4c6-179778662cf5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:48:36.564508 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.564477    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:36.564508 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.564503    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97fd8ab3-c645-439c-a4c6-179778662cf5-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:36.564681 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.564516    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:36.564681 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.564528    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxbkt\" (UniqueName: \"kubernetes.io/projected/97fd8ab3-c645-439c-a4c6-179778662cf5-kube-api-access-cxbkt\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:36.564681 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.564541    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:36.564681 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.564552    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:36.564681 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.564563    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:36.564681 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:36.564573    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97fd8ab3-c645-439c-a4c6-179778662cf5-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:48:37.003883 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:37.003846    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd3e8fd6-590a-4303-8b33-c2f369092447" path="/var/lib/kubelet/pods/fd3e8fd6-590a-4303-8b33-c2f369092447/volumes"
Sep 09 21:48:37.100739 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:37.100707    2578 generic.go:356] "Generic (PLEG): container finished" podID="97fd8ab3-c645-439c-a4c6-179778662cf5" containerID="cb376fcf0a0da52e64d39fddee1030bfa2ef3fe15496fbadd3e7b4f51dcfcd8c" exitCode=0
Sep 09 21:48:37.100887 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:37.100759    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"
Sep 09 21:48:37.100887 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:37.100759    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d" event={"ID":"97fd8ab3-c645-439c-a4c6-179778662cf5","Type":"ContainerDied","Data":"cb376fcf0a0da52e64d39fddee1030bfa2ef3fe15496fbadd3e7b4f51dcfcd8c"}
Sep 09 21:48:37.100887 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:37.100814    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d" event={"ID":"97fd8ab3-c645-439c-a4c6-179778662cf5","Type":"ContainerDied","Data":"7b08ced8407356c82f095a456ac98de5d8d9b3e071a98359b5a42e8003af127c"}
Sep 09 21:48:37.100887 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:37.100835    2578 scope.go:117] "RemoveContainer" containerID="cb376fcf0a0da52e64d39fddee1030bfa2ef3fe15496fbadd3e7b4f51dcfcd8c"
Sep 09 21:48:37.110683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:37.110663    2578 scope.go:117] "RemoveContainer" containerID="cb1ba6b37715461beeec7076e33f1f56c800582d355e8c061334ba7539134f32"
Sep 09 21:48:37.119353 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:37.119333    2578 scope.go:117] "RemoveContainer" containerID="cb376fcf0a0da52e64d39fddee1030bfa2ef3fe15496fbadd3e7b4f51dcfcd8c"
Sep 09 21:48:37.119617 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:48:37.119600    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb376fcf0a0da52e64d39fddee1030bfa2ef3fe15496fbadd3e7b4f51dcfcd8c\": container with ID starting with cb376fcf0a0da52e64d39fddee1030bfa2ef3fe15496fbadd3e7b4f51dcfcd8c not found: ID does not exist" containerID="cb376fcf0a0da52e64d39fddee1030bfa2ef3fe15496fbadd3e7b4f51dcfcd8c"
Sep 09 21:48:37.119693 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:37.119624    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb376fcf0a0da52e64d39fddee1030bfa2ef3fe15496fbadd3e7b4f51dcfcd8c"} err="failed to get container status \"cb376fcf0a0da52e64d39fddee1030bfa2ef3fe15496fbadd3e7b4f51dcfcd8c\": rpc error: code = NotFound desc = could not find container \"cb376fcf0a0da52e64d39fddee1030bfa2ef3fe15496fbadd3e7b4f51dcfcd8c\": container with ID starting with cb376fcf0a0da52e64d39fddee1030bfa2ef3fe15496fbadd3e7b4f51dcfcd8c not found: ID does not exist"
Sep 09 21:48:37.119693 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:37.119638    2578 scope.go:117] "RemoveContainer" containerID="cb1ba6b37715461beeec7076e33f1f56c800582d355e8c061334ba7539134f32"
Sep 09 21:48:37.119766 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:37.119728    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"]
Sep 09 21:48:37.119879 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:48:37.119861    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb1ba6b37715461beeec7076e33f1f56c800582d355e8c061334ba7539134f32\": container with ID starting with cb1ba6b37715461beeec7076e33f1f56c800582d355e8c061334ba7539134f32 not found: ID does not exist" containerID="cb1ba6b37715461beeec7076e33f1f56c800582d355e8c061334ba7539134f32"
Sep 09 21:48:37.119923 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:37.119885    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb1ba6b37715461beeec7076e33f1f56c800582d355e8c061334ba7539134f32"} err="failed to get container status \"cb1ba6b37715461beeec7076e33f1f56c800582d355e8c061334ba7539134f32\": rpc error: code = NotFound desc = could not find container \"cb1ba6b37715461beeec7076e33f1f56c800582d355e8c061334ba7539134f32\": container with ID starting with cb1ba6b37715461beeec7076e33f1f56c800582d355e8c061334ba7539134f32 not found: ID does not exist"
Sep 09 21:48:37.126426 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:37.126402    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-ef9b2234-openshift-default-78cff6548d-gdb5d"]
Sep 09 21:48:39.004144 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:39.004113    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97fd8ab3-c645-439c-a4c6-179778662cf5" path="/var/lib/kubelet/pods/97fd8ab3-c645-439c-a4c6-179778662cf5/volumes"
Sep 09 21:48:53.464219 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:48:53.464181    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:49:04.771521 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:04.771488    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:49:18.886595 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:18.886558    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"]
Sep 09 21:49:18.887091 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:18.886829    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk" podUID="8f7deee2-30a8-4ac4-9b6f-12cdb883e761" containerName="istio-proxy" containerID="cri-o://a835c9cadb65f4056cc2f2e266d52d4371dbfb0c313afbea657816fce0f1f337" gracePeriod=30
Sep 09 21:49:19.592998 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.592951    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"]
Sep 09 21:49:19.593552 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.593512    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97fd8ab3-c645-439c-a4c6-179778662cf5" containerName="istio-proxy"
Sep 09 21:49:19.593653 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.593555    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="97fd8ab3-c645-439c-a4c6-179778662cf5" containerName="istio-proxy"
Sep 09 21:49:19.593653 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.593583    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd3e8fd6-590a-4303-8b33-c2f369092447" containerName="revoke-keys"
Sep 09 21:49:19.593653 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.593589    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd3e8fd6-590a-4303-8b33-c2f369092447" containerName="revoke-keys"
Sep 09 21:49:19.593760 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.593657    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="97fd8ab3-c645-439c-a4c6-179778662cf5" containerName="istio-proxy"
Sep 09 21:49:19.593760 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.593666    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd3e8fd6-590a-4303-8b33-c2f369092447" containerName="revoke-keys"
Sep 09 21:49:19.596431 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.596416    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.598567 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.598548    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-331c9d-openshift-default-dockercfg-jcrld\""
Sep 09 21:49:19.608400 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.608378    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"]
Sep 09 21:49:19.741120 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.741086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-workload-certs\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.741120 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.741125    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-envoy\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.741348 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.741174    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-token\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.741348 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.741211    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c99ff219-f707-4de1-8299-c7ea5bf1d199-istiod-ca-cert\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.741348 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.741247    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kh8j\" (UniqueName: \"kubernetes.io/projected/c99ff219-f707-4de1-8299-c7ea5bf1d199-kube-api-access-4kh8j\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.741460 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.741337    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-workload-socket\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.741460 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.741380    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-podinfo\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.741460 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.741397    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-credential-socket\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.741460 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.741450    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-data\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.842884 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.842853    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-token\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.842884 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.842886    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c99ff219-f707-4de1-8299-c7ea5bf1d199-istiod-ca-cert\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.843152 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.842914    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kh8j\" (UniqueName: \"kubernetes.io/projected/c99ff219-f707-4de1-8299-c7ea5bf1d199-kube-api-access-4kh8j\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.843152 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.843004    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-workload-socket\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.843152 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.843042    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-podinfo\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.843152 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.843063    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-credential-socket\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.843152 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.843122    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-data\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.843152 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.843150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-workload-certs\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.843469 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.843181    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-envoy\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.843469 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.843435    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-workload-socket\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.843590 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.843503    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-data\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.843590 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.843568    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-workload-certs\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.843705 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.843685    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-credential-socket\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.843756 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.843699    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c99ff219-f707-4de1-8299-c7ea5bf1d199-istiod-ca-cert\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.845375 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.845357    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-envoy\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.845627 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.845605    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-podinfo\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.851236 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.851215    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kh8j\" (UniqueName: \"kubernetes.io/projected/c99ff219-f707-4de1-8299-c7ea5bf1d199-kube-api-access-4kh8j\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.851464 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.851445    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-token\") pod \"e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") " pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:19.908607 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:19.908581    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:20.183682 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:20.183658    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"]
Sep 09 21:49:20.185246 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:49:20.185218    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc99ff219_f707_4de1_8299_c7ea5bf1d199.slice/crio-b5f1025e22b3b724bd513f940d02b804a5ea23dbf783396123480b9d4c22a4b5 WatchSource:0}: Error finding container b5f1025e22b3b724bd513f940d02b804a5ea23dbf783396123480b9d4c22a4b5: Status 404 returned error can't find the container with id b5f1025e22b3b724bd513f940d02b804a5ea23dbf783396123480b9d4c22a4b5
Sep 09 21:49:20.187850 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:20.187820    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:49:20.187900 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:20.187885    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:49:20.187943 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:20.187909    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:49:20.359922 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:20.359885    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9" event={"ID":"c99ff219-f707-4de1-8299-c7ea5bf1d199","Type":"ContainerStarted","Data":"c37a9c3589053623cdd5f037f7ffd31a0151e0798bc4a55391254e43aeca38ad"}
Sep 09 21:49:20.360105 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:20.359945    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9" event={"ID":"c99ff219-f707-4de1-8299-c7ea5bf1d199","Type":"ContainerStarted","Data":"b5f1025e22b3b724bd513f940d02b804a5ea23dbf783396123480b9d4c22a4b5"}
Sep 09 21:49:20.385596 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:20.385542    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9" podStartSLOduration=1.385505924 podStartE2EDuration="1.385505924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:49:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:49:20.382458998 +0000 UTC m=+979.855423910" watchObservedRunningTime="2026-09-09 21:49:20.385505924 +0000 UTC m=+979.858470834"
Sep 09 21:49:20.909027 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:20.908988    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:21.914830 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:21.914795    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:22.371794 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:22.371768    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:22.372729 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:22.372713    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:24.140014 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.139990    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:49:24.283060 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.283036    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istiod-ca-cert\") pod \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") "
Sep 09 21:49:24.283206 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.283078    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-token\") pod \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") "
Sep 09 21:49:24.283206 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.283116    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-td2bh\" (UniqueName: \"kubernetes.io/projected/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-kube-api-access-td2bh\") pod \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") "
Sep 09 21:49:24.283206 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.283155    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-workload-socket\") pod \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") "
Sep 09 21:49:24.283206 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.283178    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-envoy\") pod \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") "
Sep 09 21:49:24.283206 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.283195    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-podinfo\") pod \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") "
Sep 09 21:49:24.283498 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.283234    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-workload-certs\") pod \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") "
Sep 09 21:49:24.283498 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.283307    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-credential-socket\") pod \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") "
Sep 09 21:49:24.283498 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.283332    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-data\") pod \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\" (UID: \"8f7deee2-30a8-4ac4-9b6f-12cdb883e761\") "
Sep 09 21:49:24.283498 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.283462    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f7deee2-30a8-4ac4-9b6f-12cdb883e761" (UID: "8f7deee2-30a8-4ac4-9b6f-12cdb883e761"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:49:24.283706 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.283613    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f7deee2-30a8-4ac4-9b6f-12cdb883e761" (UID: "8f7deee2-30a8-4ac4-9b6f-12cdb883e761"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:49:24.283706 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.283667    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f7deee2-30a8-4ac4-9b6f-12cdb883e761" (UID: "8f7deee2-30a8-4ac4-9b6f-12cdb883e761"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:49:24.283706 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.283685    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:49:24.283706 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.283692    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f7deee2-30a8-4ac4-9b6f-12cdb883e761" (UID: "8f7deee2-30a8-4ac4-9b6f-12cdb883e761"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:49:24.284009 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.283985    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f7deee2-30a8-4ac4-9b6f-12cdb883e761" (UID: "8f7deee2-30a8-4ac4-9b6f-12cdb883e761"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:49:24.285468 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.285442    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f7deee2-30a8-4ac4-9b6f-12cdb883e761" (UID: "8f7deee2-30a8-4ac4-9b6f-12cdb883e761"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:49:24.285672 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.285646    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-kube-api-access-td2bh" (OuterVolumeSpecName: "kube-api-access-td2bh") pod "8f7deee2-30a8-4ac4-9b6f-12cdb883e761" (UID: "8f7deee2-30a8-4ac4-9b6f-12cdb883e761"). InnerVolumeSpecName "kube-api-access-td2bh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:49:24.285767 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.285703    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f7deee2-30a8-4ac4-9b6f-12cdb883e761" (UID: "8f7deee2-30a8-4ac4-9b6f-12cdb883e761"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:49:24.285767 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.285717    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f7deee2-30a8-4ac4-9b6f-12cdb883e761" (UID: "8f7deee2-30a8-4ac4-9b6f-12cdb883e761"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:49:24.388227 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.388089    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:49:24.388227 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.388120    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:49:24.388227 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.388135    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:49:24.388227 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.388148    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:49:24.388227 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.388159    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-td2bh\" (UniqueName: \"kubernetes.io/projected/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-kube-api-access-td2bh\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:49:24.388227 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.388171    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:49:24.388227 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.388183    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:49:24.388227 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.388194    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f7deee2-30a8-4ac4-9b6f-12cdb883e761-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:49:24.398688 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.394858    2578 generic.go:356] "Generic (PLEG): container finished" podID="8f7deee2-30a8-4ac4-9b6f-12cdb883e761" containerID="a835c9cadb65f4056cc2f2e266d52d4371dbfb0c313afbea657816fce0f1f337" exitCode=0
Sep 09 21:49:24.398688 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.395740    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"
Sep 09 21:49:24.398688 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.398589    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk" event={"ID":"8f7deee2-30a8-4ac4-9b6f-12cdb883e761","Type":"ContainerDied","Data":"a835c9cadb65f4056cc2f2e266d52d4371dbfb0c313afbea657816fce0f1f337"}
Sep 09 21:49:24.398688 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.398644    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk" event={"ID":"8f7deee2-30a8-4ac4-9b6f-12cdb883e761","Type":"ContainerDied","Data":"da46ff4cc908a4e0266e53137b1edc93e20018bd056a92e9f436c0da633283a5"}
Sep 09 21:49:24.398688 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.398674    2578 scope.go:117] "RemoveContainer" containerID="a835c9cadb65f4056cc2f2e266d52d4371dbfb0c313afbea657816fce0f1f337"
Sep 09 21:49:24.413610 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.413591    2578 scope.go:117] "RemoveContainer" containerID="7c582e740c5995abdb150b22566f02192b75d30ef3d89462567943316c64f38a"
Sep 09 21:49:24.422386 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.422204    2578 scope.go:117] "RemoveContainer" containerID="a835c9cadb65f4056cc2f2e266d52d4371dbfb0c313afbea657816fce0f1f337"
Sep 09 21:49:24.422630 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:49:24.422604    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a835c9cadb65f4056cc2f2e266d52d4371dbfb0c313afbea657816fce0f1f337\": container with ID starting with a835c9cadb65f4056cc2f2e266d52d4371dbfb0c313afbea657816fce0f1f337 not found: ID does not exist" containerID="a835c9cadb65f4056cc2f2e266d52d4371dbfb0c313afbea657816fce0f1f337"
Sep 09 21:49:24.422693 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.422638    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a835c9cadb65f4056cc2f2e266d52d4371dbfb0c313afbea657816fce0f1f337"} err="failed to get container status \"a835c9cadb65f4056cc2f2e266d52d4371dbfb0c313afbea657816fce0f1f337\": rpc error: code = NotFound desc = could not find container \"a835c9cadb65f4056cc2f2e266d52d4371dbfb0c313afbea657816fce0f1f337\": container with ID starting with a835c9cadb65f4056cc2f2e266d52d4371dbfb0c313afbea657816fce0f1f337 not found: ID does not exist"
Sep 09 21:49:24.422693 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.422654    2578 scope.go:117] "RemoveContainer" containerID="7c582e740c5995abdb150b22566f02192b75d30ef3d89462567943316c64f38a"
Sep 09 21:49:24.422938 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:49:24.422920    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c582e740c5995abdb150b22566f02192b75d30ef3d89462567943316c64f38a\": container with ID starting with 7c582e740c5995abdb150b22566f02192b75d30ef3d89462567943316c64f38a not found: ID does not exist" containerID="7c582e740c5995abdb150b22566f02192b75d30ef3d89462567943316c64f38a"
Sep 09 21:49:24.423099 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.422949    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c582e740c5995abdb150b22566f02192b75d30ef3d89462567943316c64f38a"} err="failed to get container status \"7c582e740c5995abdb150b22566f02192b75d30ef3d89462567943316c64f38a\": rpc error: code = NotFound desc = could not find container \"7c582e740c5995abdb150b22566f02192b75d30ef3d89462567943316c64f38a\": container with ID starting with 7c582e740c5995abdb150b22566f02192b75d30ef3d89462567943316c64f38a not found: ID does not exist"
Sep 09 21:49:24.425672 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.425652    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"]
Sep 09 21:49:24.430524 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:24.430503    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-66cedd-openshift-default-66bbb95fb6-rv5zk"]
Sep 09 21:49:25.003646 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:25.003612    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f7deee2-30a8-4ac4-9b6f-12cdb883e761" path="/var/lib/kubelet/pods/8f7deee2-30a8-4ac4-9b6f-12cdb883e761/volumes"
Sep 09 21:49:29.886189 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:29.886155    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:49:38.413157 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:38.413116    2578 patch_prober.go:28] interesting pod/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 21:49:38.413639 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:38.413190    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9" podUID="c99ff219-f707-4de1-8299-c7ea5bf1d199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 21:49:41.513975 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:41.513930    2578 generic.go:356] "Generic (PLEG): container finished" podID="c99ff219-f707-4de1-8299-c7ea5bf1d199" containerID="c37a9c3589053623cdd5f037f7ffd31a0151e0798bc4a55391254e43aeca38ad" exitCode=0
Sep 09 21:49:41.514388 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:41.513981    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9" event={"ID":"c99ff219-f707-4de1-8299-c7ea5bf1d199","Type":"ContainerDied","Data":"c37a9c3589053623cdd5f037f7ffd31a0151e0798bc4a55391254e43aeca38ad"}
Sep 09 21:49:41.514388 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:41.514321    2578 scope.go:117] "RemoveContainer" containerID="c37a9c3589053623cdd5f037f7ffd31a0151e0798bc4a55391254e43aeca38ad"
Sep 09 21:49:41.515403 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:41.515372    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:49:41.515465 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:41.515443    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:49:41.515518 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:41.515473    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:49:42.522321 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:42.522287    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9" event={"ID":"c99ff219-f707-4de1-8299-c7ea5bf1d199","Type":"ContainerStarted","Data":"d16cf1f6c8c15520acc05dfcb622f7cb0f14399c73055886cc190b30ae3448e9"}
Sep 09 21:49:42.909689 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:42.909602    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:42.909838 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:42.909761    2578 patch_prober.go:28] interesting pod/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 09 21:49:42.909838 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:42.909809    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9" podUID="c99ff219-f707-4de1-8299-c7ea5bf1d199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 09 21:49:43.908879 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:43.908843    2578 patch_prober.go:28] interesting pod/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 09 21:49:43.909308 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:43.908898    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9" podUID="c99ff219-f707-4de1-8299-c7ea5bf1d199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 09 21:49:44.909510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:44.909465    2578 patch_prober.go:28] interesting pod/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 09 21:49:44.909917 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:44.909529    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9" podUID="c99ff219-f707-4de1-8299-c7ea5bf1d199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 09 21:49:45.909235 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:45.909199    2578 patch_prober.go:28] interesting pod/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 09 21:49:45.909399 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:45.909257    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9" podUID="c99ff219-f707-4de1-8299-c7ea5bf1d199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 09 21:49:46.909891 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:46.909855    2578 patch_prober.go:28] interesting pod/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 09 21:49:46.910280 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:46.909915    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9" podUID="c99ff219-f707-4de1-8299-c7ea5bf1d199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 09 21:49:47.912703 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:47.912676    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:47.913178 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:47.913141    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:47.913910 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:47.913891    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:49:49.579954 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:49:49.579924    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:50:06.781319 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.781279    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"]
Sep 09 21:50:06.781715 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.781689    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f7deee2-30a8-4ac4-9b6f-12cdb883e761" containerName="istio-proxy"
Sep 09 21:50:06.781715 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.781701    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f7deee2-30a8-4ac4-9b6f-12cdb883e761" containerName="istio-proxy"
Sep 09 21:50:06.781831 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.781717    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97fd8ab3-c645-439c-a4c6-179778662cf5" containerName="istio-proxy"
Sep 09 21:50:06.781831 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.781724    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="97fd8ab3-c645-439c-a4c6-179778662cf5" containerName="istio-proxy"
Sep 09 21:50:06.781831 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.781804    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="97fd8ab3-c645-439c-a4c6-179778662cf5" containerName="istio-proxy"
Sep 09 21:50:06.781831 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.781811    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f7deee2-30a8-4ac4-9b6f-12cdb883e761" containerName="istio-proxy"
Sep 09 21:50:06.781831 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.781817    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f7deee2-30a8-4ac4-9b6f-12cdb883e761" containerName="istio-proxy"
Sep 09 21:50:06.786162 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.786141    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.788610 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.788587    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-65d42a05-openshift-default-dockercfg-srcfk\""
Sep 09 21:50:06.801680 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.801657    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"]
Sep 09 21:50:06.889211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.889174    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdf87fbd-a936-455f-b899-88fd95e501d5-istiod-ca-cert\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.889594 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.889568    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-podinfo\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.889768 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.889752    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-workload-socket\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.889912 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.889898    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-token\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.890095 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.890077    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpfsr\" (UniqueName: \"kubernetes.io/projected/cdf87fbd-a936-455f-b899-88fd95e501d5-kube-api-access-wpfsr\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.890246 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.890230    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-data\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.890393 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.890380    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-envoy\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.890547 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.890523    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-credential-socket\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.890680 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.890597    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-workload-certs\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.991519 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.991479    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-podinfo\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.991723 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.991526    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-workload-socket\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.991723 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.991559    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-token\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.991723 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.991596    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpfsr\" (UniqueName: \"kubernetes.io/projected/cdf87fbd-a936-455f-b899-88fd95e501d5-kube-api-access-wpfsr\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.991723 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.991621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-data\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.991940 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.991760    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-envoy\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.991940 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.991811    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-credential-socket\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.991940 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.991854    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-workload-certs\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.992150 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.991941    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdf87fbd-a936-455f-b899-88fd95e501d5-istiod-ca-cert\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.992150 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.992048    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-workload-socket\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.992150 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.992084    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-data\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.992297 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.992275    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-credential-socket\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.992341 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.992320    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-workload-certs\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.992581 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.992564    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdf87fbd-a936-455f-b899-88fd95e501d5-istiod-ca-cert\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.993964 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.993934    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-envoy\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:06.994210 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:06.994190    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-podinfo\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:07.000327 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:07.000306    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-token\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:07.000446 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:07.000423    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpfsr\" (UniqueName: \"kubernetes.io/projected/cdf87fbd-a936-455f-b899-88fd95e501d5-kube-api-access-wpfsr\") pod \"e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") " pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:07.096447 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:07.096364    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:07.381267 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:07.381232    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"]
Sep 09 21:50:07.382661 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:50:07.382634    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf87fbd_a936_455f_b899_88fd95e501d5.slice/crio-47babb64d8c8ac67c85efd411d83d0048e3be3d7346c845dd7287875aff59982 WatchSource:0}: Error finding container 47babb64d8c8ac67c85efd411d83d0048e3be3d7346c845dd7287875aff59982: Status 404 returned error can't find the container with id 47babb64d8c8ac67c85efd411d83d0048e3be3d7346c845dd7287875aff59982
Sep 09 21:50:07.385447 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:07.385416    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:50:07.385542 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:07.385498    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:50:07.385542 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:07.385529    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:50:07.683433 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:07.683343    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5" event={"ID":"cdf87fbd-a936-455f-b899-88fd95e501d5","Type":"ContainerStarted","Data":"9f0d3703fd661bf3036d13bb0f4cb880ccb9c2dff36f0c282ecae970ea4ca0aa"}
Sep 09 21:50:07.683433 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:07.683401    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5" event={"ID":"cdf87fbd-a936-455f-b899-88fd95e501d5","Type":"ContainerStarted","Data":"47babb64d8c8ac67c85efd411d83d0048e3be3d7346c845dd7287875aff59982"}
Sep 09 21:50:07.705442 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:07.705398    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5" podStartSLOduration=1.705381338 podStartE2EDuration="1.705381338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:50:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:50:07.705087543 +0000 UTC m=+1027.178052474" watchObservedRunningTime="2026-09-09 21:50:07.705381338 +0000 UTC m=+1027.178346250"
Sep 09 21:50:08.097177 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:08.097150    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:08.102072 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:08.102049    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:08.689821 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:08.689783    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:08.690801 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:08.690782    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:13.783859 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:13.783821    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"]
Sep 09 21:50:13.785258 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:13.785233    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f7deee2-30a8-4ac4-9b6f-12cdb883e761" containerName="istio-proxy"
Sep 09 21:50:13.785406 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:13.785394    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f7deee2-30a8-4ac4-9b6f-12cdb883e761" containerName="istio-proxy"
Sep 09 21:50:13.790107 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:13.790085    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:13.793029 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:13.793006    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-65d42a05-serving-cert\""
Sep 09 21:50:13.795401 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:13.795377    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"]
Sep 09 21:50:13.959867 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:13.959833    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b9e3ea-03be-4b64-8889-35049d98b88e-openshift-service-ca\") pod \"maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:13.960096 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:13.959904    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b9e3ea-03be-4b64-8889-35049d98b88e-maas-api-tls\") pod \"maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:13.960096 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:13.959936    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58b9e3ea-03be-4b64-8889-35049d98b88e-metrics-certs\") pod \"maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:13.960096 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:13.959962    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dfzq\" (UniqueName: \"kubernetes.io/projected/58b9e3ea-03be-4b64-8889-35049d98b88e-kube-api-access-8dfzq\") pod \"maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:14.061177 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:14.061101    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b9e3ea-03be-4b64-8889-35049d98b88e-maas-api-tls\") pod \"maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:14.061177 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:14.061151    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58b9e3ea-03be-4b64-8889-35049d98b88e-metrics-certs\") pod \"maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:14.061347 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:14.061193    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dfzq\" (UniqueName: \"kubernetes.io/projected/58b9e3ea-03be-4b64-8889-35049d98b88e-kube-api-access-8dfzq\") pod \"maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:14.061347 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:14.061283    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b9e3ea-03be-4b64-8889-35049d98b88e-openshift-service-ca\") pod \"maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:14.061906 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:14.061887    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b9e3ea-03be-4b64-8889-35049d98b88e-openshift-service-ca\") pod \"maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:14.063585 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:14.063558    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b9e3ea-03be-4b64-8889-35049d98b88e-maas-api-tls\") pod \"maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:14.063723 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:14.063705    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58b9e3ea-03be-4b64-8889-35049d98b88e-metrics-certs\") pod \"maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:14.074585 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:14.074562    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dfzq\" (UniqueName: \"kubernetes.io/projected/58b9e3ea-03be-4b64-8889-35049d98b88e-kube-api-access-8dfzq\") pod \"maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:14.102326 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:14.102301    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:14.377655 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:14.377627    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"]
Sep 09 21:50:14.379078 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:50:14.379054    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58b9e3ea_03be_4b64_8889_35049d98b88e.slice/crio-e880cf25d0f060d614a3843c21f2b2b5ecac337e2dfc6a59c438ff8da2ac69eb WatchSource:0}: Error finding container e880cf25d0f060d614a3843c21f2b2b5ecac337e2dfc6a59c438ff8da2ac69eb: Status 404 returned error can't find the container with id e880cf25d0f060d614a3843c21f2b2b5ecac337e2dfc6a59c438ff8da2ac69eb
Sep 09 21:50:14.380425 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:14.380409    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 21:50:14.724244 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:14.724162    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd" event={"ID":"58b9e3ea-03be-4b64-8889-35049d98b88e","Type":"ContainerStarted","Data":"e880cf25d0f060d614a3843c21f2b2b5ecac337e2dfc6a59c438ff8da2ac69eb"}
Sep 09 21:50:15.731468 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:15.731416    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd" event={"ID":"58b9e3ea-03be-4b64-8889-35049d98b88e","Type":"ContainerStarted","Data":"7a6c18345c29cffdc67dd3d9a398a5d88efc09dcb999e900bed4687736180ad5"}
Sep 09 21:50:15.731950 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:15.731487    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:15.749420 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:15.749372    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd" podStartSLOduration=2.3047910910000002 podStartE2EDuration="2.749356798s" totalImagesPullingTime="444.565707ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:50:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:50:15.748607873 +0000 UTC m=+1035.221572784" watchObservedRunningTime="2026-09-09 21:50:15.749356798 +0000 UTC m=+1035.222321707"
Sep 09 21:50:21.742371 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:21.742339    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:25.665736 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:25.665699    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:50:36.872823 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:36.872792    2578 generic.go:356] "Generic (PLEG): container finished" podID="cdf87fbd-a936-455f-b899-88fd95e501d5" containerID="9f0d3703fd661bf3036d13bb0f4cb880ccb9c2dff36f0c282ecae970ea4ca0aa" exitCode=0
Sep 09 21:50:36.873239 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:36.872860    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5" event={"ID":"cdf87fbd-a936-455f-b899-88fd95e501d5","Type":"ContainerDied","Data":"9f0d3703fd661bf3036d13bb0f4cb880ccb9c2dff36f0c282ecae970ea4ca0aa"}
Sep 09 21:50:36.873284 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:36.873264    2578 scope.go:117] "RemoveContainer" containerID="9f0d3703fd661bf3036d13bb0f4cb880ccb9c2dff36f0c282ecae970ea4ca0aa"
Sep 09 21:50:36.874340 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:36.874310    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:50:36.874456 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:36.874375    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:50:36.874456 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:36.874406    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:50:37.880499 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:37.880462    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5" event={"ID":"cdf87fbd-a936-455f-b899-88fd95e501d5","Type":"ContainerStarted","Data":"ad714d52676a169f1922c249ca3f9ab7f443e0bb3909811767e0eb8757bd5b7d"}
Sep 09 21:50:38.096855 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:38.096817    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:38.098165 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:38.098139    2578 patch_prober.go:28] interesting pod/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 09 21:50:38.098273 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:38.098189    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5" podUID="cdf87fbd-a936-455f-b899-88fd95e501d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 09 21:50:38.690306 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:38.690224    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:39.097434 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:39.097396    2578 patch_prober.go:28] interesting pod/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 09 21:50:39.097829 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:39.097456    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5" podUID="cdf87fbd-a936-455f-b899-88fd95e501d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 09 21:50:40.096948 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:40.096910    2578 patch_prober.go:28] interesting pod/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 09 21:50:40.097164 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:40.096994    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5" podUID="cdf87fbd-a936-455f-b899-88fd95e501d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 09 21:50:41.097699 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:41.097663    2578 patch_prober.go:28] interesting pod/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 09 21:50:41.098109 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:41.097715    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5" podUID="cdf87fbd-a936-455f-b899-88fd95e501d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 09 21:50:42.097232 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:42.097197    2578 patch_prober.go:28] interesting pod/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 09 21:50:42.097408 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:42.097247    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5" podUID="cdf87fbd-a936-455f-b899-88fd95e501d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 09 21:50:43.097116 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:43.097074    2578 patch_prober.go:28] interesting pod/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 09 21:50:43.097602 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:43.097142    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5" podUID="cdf87fbd-a936-455f-b899-88fd95e501d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 09 21:50:44.101107 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:44.101077    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:44.102092 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:44.102070    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:50:55.210585 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.210503    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"]
Sep 09 21:50:55.211024 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.210798    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd" podUID="58b9e3ea-03be-4b64-8889-35049d98b88e" containerName="maas-api" containerID="cri-o://7a6c18345c29cffdc67dd3d9a398a5d88efc09dcb999e900bed4687736180ad5" gracePeriod=30
Sep 09 21:50:55.452549 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.452527    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:55.560279 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.560255    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dfzq\" (UniqueName: \"kubernetes.io/projected/58b9e3ea-03be-4b64-8889-35049d98b88e-kube-api-access-8dfzq\") pod \"58b9e3ea-03be-4b64-8889-35049d98b88e\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") "
Sep 09 21:50:55.560436 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.560371    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b9e3ea-03be-4b64-8889-35049d98b88e-maas-api-tls\") pod \"58b9e3ea-03be-4b64-8889-35049d98b88e\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") "
Sep 09 21:50:55.560436 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.560402    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58b9e3ea-03be-4b64-8889-35049d98b88e-metrics-certs\") pod \"58b9e3ea-03be-4b64-8889-35049d98b88e\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") "
Sep 09 21:50:55.560436 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.560429    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b9e3ea-03be-4b64-8889-35049d98b88e-openshift-service-ca\") pod \"58b9e3ea-03be-4b64-8889-35049d98b88e\" (UID: \"58b9e3ea-03be-4b64-8889-35049d98b88e\") "
Sep 09 21:50:55.560886 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.560855    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58b9e3ea-03be-4b64-8889-35049d98b88e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "58b9e3ea-03be-4b64-8889-35049d98b88e" (UID: "58b9e3ea-03be-4b64-8889-35049d98b88e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:50:55.562615 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.562581    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58b9e3ea-03be-4b64-8889-35049d98b88e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "58b9e3ea-03be-4b64-8889-35049d98b88e" (UID: "58b9e3ea-03be-4b64-8889-35049d98b88e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:50:55.562849 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.562815    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58b9e3ea-03be-4b64-8889-35049d98b88e-kube-api-access-8dfzq" (OuterVolumeSpecName: "kube-api-access-8dfzq") pod "58b9e3ea-03be-4b64-8889-35049d98b88e" (UID: "58b9e3ea-03be-4b64-8889-35049d98b88e"). InnerVolumeSpecName "kube-api-access-8dfzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:50:55.562932 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.562871    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58b9e3ea-03be-4b64-8889-35049d98b88e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "58b9e3ea-03be-4b64-8889-35049d98b88e" (UID: "58b9e3ea-03be-4b64-8889-35049d98b88e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:50:55.661416 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.661367    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58b9e3ea-03be-4b64-8889-35049d98b88e-maas-api-tls\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:50:55.661416 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.661399    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58b9e3ea-03be-4b64-8889-35049d98b88e-metrics-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:50:55.661416 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.661410    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58b9e3ea-03be-4b64-8889-35049d98b88e-openshift-service-ca\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:50:55.661416 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.661421    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dfzq\" (UniqueName: \"kubernetes.io/projected/58b9e3ea-03be-4b64-8889-35049d98b88e-kube-api-access-8dfzq\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:50:55.994518 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.994433    2578 generic.go:356] "Generic (PLEG): container finished" podID="58b9e3ea-03be-4b64-8889-35049d98b88e" containerID="7a6c18345c29cffdc67dd3d9a398a5d88efc09dcb999e900bed4687736180ad5" exitCode=0
Sep 09 21:50:55.994518 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.994483    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"
Sep 09 21:50:55.994698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.994517    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd" event={"ID":"58b9e3ea-03be-4b64-8889-35049d98b88e","Type":"ContainerDied","Data":"7a6c18345c29cffdc67dd3d9a398a5d88efc09dcb999e900bed4687736180ad5"}
Sep 09 21:50:55.994698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.994574    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd" event={"ID":"58b9e3ea-03be-4b64-8889-35049d98b88e","Type":"ContainerDied","Data":"e880cf25d0f060d614a3843c21f2b2b5ecac337e2dfc6a59c438ff8da2ac69eb"}
Sep 09 21:50:55.994698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:55.994596    2578 scope.go:117] "RemoveContainer" containerID="7a6c18345c29cffdc67dd3d9a398a5d88efc09dcb999e900bed4687736180ad5"
Sep 09 21:50:56.004447 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:56.004430    2578 scope.go:117] "RemoveContainer" containerID="7a6c18345c29cffdc67dd3d9a398a5d88efc09dcb999e900bed4687736180ad5"
Sep 09 21:50:56.004705 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:50:56.004684    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a6c18345c29cffdc67dd3d9a398a5d88efc09dcb999e900bed4687736180ad5\": container with ID starting with 7a6c18345c29cffdc67dd3d9a398a5d88efc09dcb999e900bed4687736180ad5 not found: ID does not exist" containerID="7a6c18345c29cffdc67dd3d9a398a5d88efc09dcb999e900bed4687736180ad5"
Sep 09 21:50:56.004745 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:56.004712    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a6c18345c29cffdc67dd3d9a398a5d88efc09dcb999e900bed4687736180ad5"} err="failed to get container status \"7a6c18345c29cffdc67dd3d9a398a5d88efc09dcb999e900bed4687736180ad5\": rpc error: code = NotFound desc = could not find container \"7a6c18345c29cffdc67dd3d9a398a5d88efc09dcb999e900bed4687736180ad5\": container with ID starting with 7a6c18345c29cffdc67dd3d9a398a5d88efc09dcb999e900bed4687736180ad5 not found: ID does not exist"
Sep 09 21:50:56.018430 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:56.018400    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"]
Sep 09 21:50:56.022201 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:56.022175    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-65d42a05-5d5f645dcf-9lbrd"]
Sep 09 21:50:57.003418 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:50:57.003390    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58b9e3ea-03be-4b64-8889-35049d98b88e" path="/var/lib/kubelet/pods/58b9e3ea-03be-4b64-8889-35049d98b88e/volumes"
Sep 09 21:51:01.698059 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.698022    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"]
Sep 09 21:51:01.698483 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.698469    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58b9e3ea-03be-4b64-8889-35049d98b88e" containerName="maas-api"
Sep 09 21:51:01.698528 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.698484    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="58b9e3ea-03be-4b64-8889-35049d98b88e" containerName="maas-api"
Sep 09 21:51:01.698568 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.698559    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="58b9e3ea-03be-4b64-8889-35049d98b88e" containerName="maas-api"
Sep 09 21:51:01.702850 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.702832    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.705201 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.705177    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-331c9d\"/\"openshift-service-ca.crt\""
Sep 09 21:51:01.705305 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.705189    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-331c9d\"/\"kube-root-ca.crt\""
Sep 09 21:51:01.706274 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.706251    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-331c9d\"/\"default-dockercfg-m68sw\""
Sep 09 21:51:01.706460 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.706433    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-331c9d\"/\"explicit-model-331c9d-kserve-self-signed-certs\""
Sep 09 21:51:01.709657 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.709637    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"]
Sep 09 21:51:01.715737 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.715712    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-tmp-dir\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.715947 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.715925    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-dshm\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.716222 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.716177    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqsfk\" (UniqueName: \"kubernetes.io/projected/0ce57c82-c33d-41c0-98ba-983ec35fa28f-kube-api-access-gqsfk\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.716332 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.716231    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-home\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.716332 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.716260    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ce57c82-c33d-41c0-98ba-983ec35fa28f-tls-certs\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.716449 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.716332    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-model-cache\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.740148 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.740117    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"]
Sep 09 21:51:01.740384 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.740342    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5" podUID="cdf87fbd-a936-455f-b899-88fd95e501d5" containerName="istio-proxy" containerID="cri-o://ad714d52676a169f1922c249ca3f9ab7f443e0bb3909811767e0eb8757bd5b7d" gracePeriod=30
Sep 09 21:51:01.817249 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.817217    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-tmp-dir\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.817249 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.817249    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-dshm\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.817463 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.817302    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqsfk\" (UniqueName: \"kubernetes.io/projected/0ce57c82-c33d-41c0-98ba-983ec35fa28f-kube-api-access-gqsfk\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.817463 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.817326    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-home\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.817463 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.817348    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ce57c82-c33d-41c0-98ba-983ec35fa28f-tls-certs\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.817463 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.817421    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-model-cache\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.817678 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.817604    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-tmp-dir\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.817737 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.817677    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-home\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.817778 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.817756    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-model-cache\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.819530 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.819504    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-dshm\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.819698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.819683    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ce57c82-c33d-41c0-98ba-983ec35fa28f-tls-certs\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:01.833804 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:01.833776    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqsfk\" (UniqueName: \"kubernetes.io/projected/0ce57c82-c33d-41c0-98ba-983ec35fa28f-kube-api-access-gqsfk\") pod \"explicit-model-331c9d-kserve-5b76d59f9f-26xdx\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") " pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:02.014013 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:02.013907    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:02.320310 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:02.319609    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"]
Sep 09 21:51:02.482807 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:02.482772    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:51:03.040583 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:03.040547    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx" event={"ID":"0ce57c82-c33d-41c0-98ba-983ec35fa28f","Type":"ContainerStarted","Data":"495071bf5629b75c930c46216d1fbd508d2e094428944a57c47b3f934d9f4b73"}
Sep 09 21:51:03.040583 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:03.040590    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx" event={"ID":"0ce57c82-c33d-41c0-98ba-983ec35fa28f","Type":"ContainerStarted","Data":"dec21f31148c6fd87bc43a9dacbe77b3cfbf14e8aae83a94ceeef740917f95f9"}
Sep 09 21:51:03.061201 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:03.061150    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx" podStartSLOduration=2.061135499 podStartE2EDuration="2.061135499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:51:03.058214231 +0000 UTC m=+1082.531179167" watchObservedRunningTime="2026-09-09 21:51:03.061135499 +0000 UTC m=+1082.534100408"
Sep 09 21:51:07.002209 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.002186    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:51:07.065578 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.065549    2578 generic.go:356] "Generic (PLEG): container finished" podID="cdf87fbd-a936-455f-b899-88fd95e501d5" containerID="ad714d52676a169f1922c249ca3f9ab7f443e0bb3909811767e0eb8757bd5b7d" exitCode=0
Sep 09 21:51:07.065744 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.065598    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"
Sep 09 21:51:07.065744 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.065634    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5" event={"ID":"cdf87fbd-a936-455f-b899-88fd95e501d5","Type":"ContainerDied","Data":"ad714d52676a169f1922c249ca3f9ab7f443e0bb3909811767e0eb8757bd5b7d"}
Sep 09 21:51:07.065744 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.065692    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5" event={"ID":"cdf87fbd-a936-455f-b899-88fd95e501d5","Type":"ContainerDied","Data":"47babb64d8c8ac67c85efd411d83d0048e3be3d7346c845dd7287875aff59982"}
Sep 09 21:51:07.065744 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.065723    2578 scope.go:117] "RemoveContainer" containerID="ad714d52676a169f1922c249ca3f9ab7f443e0bb3909811767e0eb8757bd5b7d"
Sep 09 21:51:07.067941 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.067918    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-data\") pod \"cdf87fbd-a936-455f-b899-88fd95e501d5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") "
Sep 09 21:51:07.068081 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.067999    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-token\") pod \"cdf87fbd-a936-455f-b899-88fd95e501d5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") "
Sep 09 21:51:07.068081 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.068035    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-podinfo\") pod \"cdf87fbd-a936-455f-b899-88fd95e501d5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") "
Sep 09 21:51:07.068081 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.068071    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-envoy\") pod \"cdf87fbd-a936-455f-b899-88fd95e501d5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") "
Sep 09 21:51:07.068255 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.068091    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-workload-socket\") pod \"cdf87fbd-a936-455f-b899-88fd95e501d5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") "
Sep 09 21:51:07.068255 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.068159    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpfsr\" (UniqueName: \"kubernetes.io/projected/cdf87fbd-a936-455f-b899-88fd95e501d5-kube-api-access-wpfsr\") pod \"cdf87fbd-a936-455f-b899-88fd95e501d5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") "
Sep 09 21:51:07.068363 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.068225    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-data" (OuterVolumeSpecName: "istio-data") pod "cdf87fbd-a936-455f-b899-88fd95e501d5" (UID: "cdf87fbd-a936-455f-b899-88fd95e501d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:51:07.068363 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.068297    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdf87fbd-a936-455f-b899-88fd95e501d5-istiod-ca-cert\") pod \"cdf87fbd-a936-455f-b899-88fd95e501d5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") "
Sep 09 21:51:07.068363 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.068326    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-workload-certs\") pod \"cdf87fbd-a936-455f-b899-88fd95e501d5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") "
Sep 09 21:51:07.068526 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.068368    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-credential-socket\") pod \"cdf87fbd-a936-455f-b899-88fd95e501d5\" (UID: \"cdf87fbd-a936-455f-b899-88fd95e501d5\") "
Sep 09 21:51:07.068698 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.068667    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdf87fbd-a936-455f-b899-88fd95e501d5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cdf87fbd-a936-455f-b899-88fd95e501d5" (UID: "cdf87fbd-a936-455f-b899-88fd95e501d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:51:07.068888 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.068670    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cdf87fbd-a936-455f-b899-88fd95e501d5" (UID: "cdf87fbd-a936-455f-b899-88fd95e501d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:51:07.068888 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.068865    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdf87fbd-a936-455f-b899-88fd95e501d5-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:51:07.068888 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.068885    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:51:07.069115 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.068897    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:51:07.069176 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.069137    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cdf87fbd-a936-455f-b899-88fd95e501d5" (UID: "cdf87fbd-a936-455f-b899-88fd95e501d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:51:07.069622 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.069599    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cdf87fbd-a936-455f-b899-88fd95e501d5" (UID: "cdf87fbd-a936-455f-b899-88fd95e501d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:51:07.070600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.070568    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cdf87fbd-a936-455f-b899-88fd95e501d5" (UID: "cdf87fbd-a936-455f-b899-88fd95e501d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:51:07.070600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.070589    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cdf87fbd-a936-455f-b899-88fd95e501d5" (UID: "cdf87fbd-a936-455f-b899-88fd95e501d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:51:07.070838 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.070821    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf87fbd-a936-455f-b899-88fd95e501d5-kube-api-access-wpfsr" (OuterVolumeSpecName: "kube-api-access-wpfsr") pod "cdf87fbd-a936-455f-b899-88fd95e501d5" (UID: "cdf87fbd-a936-455f-b899-88fd95e501d5"). InnerVolumeSpecName "kube-api-access-wpfsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:51:07.071356 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.071339    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-token" (OuterVolumeSpecName: "istio-token") pod "cdf87fbd-a936-455f-b899-88fd95e501d5" (UID: "cdf87fbd-a936-455f-b899-88fd95e501d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:51:07.091878 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.091854    2578 scope.go:117] "RemoveContainer" containerID="9f0d3703fd661bf3036d13bb0f4cb880ccb9c2dff36f0c282ecae970ea4ca0aa"
Sep 09 21:51:07.100391 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.100374    2578 scope.go:117] "RemoveContainer" containerID="ad714d52676a169f1922c249ca3f9ab7f443e0bb3909811767e0eb8757bd5b7d"
Sep 09 21:51:07.100648 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:51:07.100633    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad714d52676a169f1922c249ca3f9ab7f443e0bb3909811767e0eb8757bd5b7d\": container with ID starting with ad714d52676a169f1922c249ca3f9ab7f443e0bb3909811767e0eb8757bd5b7d not found: ID does not exist" containerID="ad714d52676a169f1922c249ca3f9ab7f443e0bb3909811767e0eb8757bd5b7d"
Sep 09 21:51:07.100699 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.100660    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad714d52676a169f1922c249ca3f9ab7f443e0bb3909811767e0eb8757bd5b7d"} err="failed to get container status \"ad714d52676a169f1922c249ca3f9ab7f443e0bb3909811767e0eb8757bd5b7d\": rpc error: code = NotFound desc = could not find container \"ad714d52676a169f1922c249ca3f9ab7f443e0bb3909811767e0eb8757bd5b7d\": container with ID starting with ad714d52676a169f1922c249ca3f9ab7f443e0bb3909811767e0eb8757bd5b7d not found: ID does not exist"
Sep 09 21:51:07.100699 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.100676    2578 scope.go:117] "RemoveContainer" containerID="9f0d3703fd661bf3036d13bb0f4cb880ccb9c2dff36f0c282ecae970ea4ca0aa"
Sep 09 21:51:07.100941 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:51:07.100923    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f0d3703fd661bf3036d13bb0f4cb880ccb9c2dff36f0c282ecae970ea4ca0aa\": container with ID starting with 9f0d3703fd661bf3036d13bb0f4cb880ccb9c2dff36f0c282ecae970ea4ca0aa not found: ID does not exist" containerID="9f0d3703fd661bf3036d13bb0f4cb880ccb9c2dff36f0c282ecae970ea4ca0aa"
Sep 09 21:51:07.101007 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.100949    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f0d3703fd661bf3036d13bb0f4cb880ccb9c2dff36f0c282ecae970ea4ca0aa"} err="failed to get container status \"9f0d3703fd661bf3036d13bb0f4cb880ccb9c2dff36f0c282ecae970ea4ca0aa\": rpc error: code = NotFound desc = could not find container \"9f0d3703fd661bf3036d13bb0f4cb880ccb9c2dff36f0c282ecae970ea4ca0aa\": container with ID starting with 9f0d3703fd661bf3036d13bb0f4cb880ccb9c2dff36f0c282ecae970ea4ca0aa not found: ID does not exist"
Sep 09 21:51:07.170424 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.170394    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:51:07.170424 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.170419    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:51:07.170424 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.170428    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:51:07.170618 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.170436    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:51:07.170618 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.170445    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf87fbd-a936-455f-b899-88fd95e501d5-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:51:07.170618 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.170453    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpfsr\" (UniqueName: \"kubernetes.io/projected/cdf87fbd-a936-455f-b899-88fd95e501d5-kube-api-access-wpfsr\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:51:07.397616 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.391318    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"]
Sep 09 21:51:07.399126 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:07.399098    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-65d42a05-openshift-default-657799b5b4-7rmn5"]
Sep 09 21:51:09.003404 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:09.003374    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdf87fbd-a936-455f-b899-88fd95e501d5" path="/var/lib/kubelet/pods/cdf87fbd-a936-455f-b899-88fd95e501d5/volumes"
Sep 09 21:51:12.014420 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:12.014388    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:12.014813 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:12.014429    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:12.027067 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:12.027044    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:12.107378 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:12.107353    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:51:15.266143 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.266108    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"]
Sep 09 21:51:15.266776 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.266756    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdf87fbd-a936-455f-b899-88fd95e501d5" containerName="istio-proxy"
Sep 09 21:51:15.266846 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.266780    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf87fbd-a936-455f-b899-88fd95e501d5" containerName="istio-proxy"
Sep 09 21:51:15.266846 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.266796    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdf87fbd-a936-455f-b899-88fd95e501d5" containerName="istio-proxy"
Sep 09 21:51:15.266846 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.266806    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf87fbd-a936-455f-b899-88fd95e501d5" containerName="istio-proxy"
Sep 09 21:51:15.266950 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.266909    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdf87fbd-a936-455f-b899-88fd95e501d5" containerName="istio-proxy"
Sep 09 21:51:15.266950 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.266926    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdf87fbd-a936-455f-b899-88fd95e501d5" containerName="istio-proxy"
Sep 09 21:51:15.271886 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.271866    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.274121 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.274098    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-d99c83-openshift-default-dockercfg-cmk7n\""
Sep 09 21:51:15.279254 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.279231    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"]
Sep 09 21:51:15.344681 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.344653    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-workload-certs\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.344769 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.344693    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-workload-socket\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.344769 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.344735    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23d09d41-47e5-4384-b6d1-df3b4087e544-istiod-ca-cert\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.344769 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.344759    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-credential-socket\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.344887 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.344775    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h6kk\" (UniqueName: \"kubernetes.io/projected/23d09d41-47e5-4384-b6d1-df3b4087e544-kube-api-access-5h6kk\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.344887 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.344802    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-envoy\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.344887 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.344875    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-token\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.345011 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.344896    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-podinfo\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.345011 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.344924    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-data\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.445829 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.445793    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-credential-socket\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.445829 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.445833    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h6kk\" (UniqueName: \"kubernetes.io/projected/23d09d41-47e5-4384-b6d1-df3b4087e544-kube-api-access-5h6kk\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.446080 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.445865    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-envoy\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.446080 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.445954    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-token\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.446080 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.445992    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-podinfo\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.446080 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.446037    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-data\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.446294 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.446089    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-workload-certs\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.446294 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.446133    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-workload-socket\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.446294 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.446187    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23d09d41-47e5-4384-b6d1-df3b4087e544-istiod-ca-cert\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.446458 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.446441    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-workload-certs\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.446520 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.446456    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-credential-socket\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.446578 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.446527    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-data\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.446634 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.446595    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-workload-socket\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.447044 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.447020    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23d09d41-47e5-4384-b6d1-df3b4087e544-istiod-ca-cert\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.448203 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.448184    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-envoy\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.448605 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.448586    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-podinfo\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.453941 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.453914    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h6kk\" (UniqueName: \"kubernetes.io/projected/23d09d41-47e5-4384-b6d1-df3b4087e544-kube-api-access-5h6kk\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.454238 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.454217    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-token\") pod \"e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") " pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.584171 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.584075    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:15.860940 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.860916    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"]
Sep 09 21:51:15.861783 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:51:15.861763    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23d09d41_47e5_4384_b6d1_df3b4087e544.slice/crio-1cce8cf2669c029d962afaa1e99c60d8a68ed6b29b79a6640cf7768cd30096ff WatchSource:0}: Error finding container 1cce8cf2669c029d962afaa1e99c60d8a68ed6b29b79a6640cf7768cd30096ff: Status 404 returned error can't find the container with id 1cce8cf2669c029d962afaa1e99c60d8a68ed6b29b79a6640cf7768cd30096ff
Sep 09 21:51:15.864314 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.864282    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:51:15.864366 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.864346    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:51:15.864408 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:15.864393    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:51:16.125491 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:16.125395    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg" event={"ID":"23d09d41-47e5-4384-b6d1-df3b4087e544","Type":"ContainerStarted","Data":"e28302ee4f3bc3ecb8909d0661414bc049d3810ac05e0bc56e5bd32d7d0765ca"}
Sep 09 21:51:16.125491 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:16.125456    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg" event={"ID":"23d09d41-47e5-4384-b6d1-df3b4087e544","Type":"ContainerStarted","Data":"1cce8cf2669c029d962afaa1e99c60d8a68ed6b29b79a6640cf7768cd30096ff"}
Sep 09 21:51:16.146738 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:16.146677    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg" podStartSLOduration=1.146661816 podStartE2EDuration="1.146661816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:51:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:51:16.143814937 +0000 UTC m=+1095.616779848" watchObservedRunningTime="2026-09-09 21:51:16.146661816 +0000 UTC m=+1095.619626767"
Sep 09 21:51:16.584707 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:16.584675    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:16.590001 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:16.589957    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:17.131749 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:17.131716    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:17.132679 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:17.132655    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:19.166669 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.166639    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"]
Sep 09 21:51:19.170546 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.170524    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:19.172805 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.172779    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-d99c83-serving-cert\""
Sep 09 21:51:19.183026 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.183001    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"]
Sep 09 21:51:19.284082 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.284040    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ee6a3c9-5b6a-4647-9771-0691097c2e35-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:19.284636 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.284603    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee6a3c9-5b6a-4647-9771-0691097c2e35-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:19.285065 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.285024    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee6a3c9-5b6a-4647-9771-0691097c2e35-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:19.285260 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.285241    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbhgz\" (UniqueName: \"kubernetes.io/projected/6ee6a3c9-5b6a-4647-9771-0691097c2e35-kube-api-access-bbhgz\") pod \"maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:19.386690 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.386662    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee6a3c9-5b6a-4647-9771-0691097c2e35-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:19.386868 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.386700    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee6a3c9-5b6a-4647-9771-0691097c2e35-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:19.386868 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.386728    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbhgz\" (UniqueName: \"kubernetes.io/projected/6ee6a3c9-5b6a-4647-9771-0691097c2e35-kube-api-access-bbhgz\") pod \"maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:19.386868 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.386795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ee6a3c9-5b6a-4647-9771-0691097c2e35-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:19.387394 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.387370    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee6a3c9-5b6a-4647-9771-0691097c2e35-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:19.389254 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.389236    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ee6a3c9-5b6a-4647-9771-0691097c2e35-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:19.389415 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.389393    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee6a3c9-5b6a-4647-9771-0691097c2e35-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:19.394610 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.394591    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbhgz\" (UniqueName: \"kubernetes.io/projected/6ee6a3c9-5b6a-4647-9771-0691097c2e35-kube-api-access-bbhgz\") pod \"maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:19.482652 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.482599    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:19.772369 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:19.772338    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"]
Sep 09 21:51:19.772910 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:51:19.772887    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ee6a3c9_5b6a_4647_9771_0691097c2e35.slice/crio-38076f2d6b619a10a2ee5be6b4663c236aa54c3682f54d39c96fb62f2bcf58da WatchSource:0}: Error finding container 38076f2d6b619a10a2ee5be6b4663c236aa54c3682f54d39c96fb62f2bcf58da: Status 404 returned error can't find the container with id 38076f2d6b619a10a2ee5be6b4663c236aa54c3682f54d39c96fb62f2bcf58da
Sep 09 21:51:20.153274 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:20.153192    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x" event={"ID":"6ee6a3c9-5b6a-4647-9771-0691097c2e35","Type":"ContainerStarted","Data":"38076f2d6b619a10a2ee5be6b4663c236aa54c3682f54d39c96fb62f2bcf58da"}
Sep 09 21:51:21.160551 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:21.160512    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x" event={"ID":"6ee6a3c9-5b6a-4647-9771-0691097c2e35","Type":"ContainerStarted","Data":"df05e6e3c25db0ff04d6f725f737873bc7aacc9c68b6c471fe6576c5f7c311c1"}
Sep 09 21:51:21.160932 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:21.160667    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:21.180451 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:21.180405    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x" podStartSLOduration=1.7015151469999998 podStartE2EDuration="2.18039214s" totalImagesPullingTime="478.876993ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:51:21.176909947 +0000 UTC m=+1100.649874870" watchObservedRunningTime="2026-09-09 21:51:21.18039214 +0000 UTC m=+1100.653357085"
Sep 09 21:51:26.467700 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:26.467666    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:51:27.171525 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:27.171495    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:51:37.294093 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:37.294067    2578 generic.go:356] "Generic (PLEG): container finished" podID="23d09d41-47e5-4384-b6d1-df3b4087e544" containerID="e28302ee4f3bc3ecb8909d0661414bc049d3810ac05e0bc56e5bd32d7d0765ca" exitCode=0
Sep 09 21:51:37.294422 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:37.294142    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg" event={"ID":"23d09d41-47e5-4384-b6d1-df3b4087e544","Type":"ContainerDied","Data":"e28302ee4f3bc3ecb8909d0661414bc049d3810ac05e0bc56e5bd32d7d0765ca"}
Sep 09 21:51:37.294566 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:37.294551    2578 scope.go:117] "RemoveContainer" containerID="e28302ee4f3bc3ecb8909d0661414bc049d3810ac05e0bc56e5bd32d7d0765ca"
Sep 09 21:51:37.295680 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:37.295651    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:51:37.295782 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:37.295736    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:51:37.295851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:37.295784    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:51:38.302579 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:38.302535    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg" event={"ID":"23d09d41-47e5-4384-b6d1-df3b4087e544","Type":"ContainerStarted","Data":"5ab708ef4db0a607e237a4e67ecd7d1bbe87fa6372824bc0a44859948e1b4acb"}
Sep 09 21:51:38.584465 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:38.584374    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:38.584615 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:38.584585    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 09 21:51:38.584658 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:38.584642    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg" podUID="23d09d41-47e5-4384-b6d1-df3b4087e544" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 09 21:51:39.584477 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:39.584441    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 09 21:51:39.584869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:39.584504    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg" podUID="23d09d41-47e5-4384-b6d1-df3b4087e544" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 09 21:51:40.585367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:40.585326    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 09 21:51:40.585855 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:40.585388    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg" podUID="23d09d41-47e5-4384-b6d1-df3b4087e544" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 09 21:51:41.585047 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:41.585011    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 09 21:51:41.585210 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:41.585080    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg" podUID="23d09d41-47e5-4384-b6d1-df3b4087e544" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 09 21:51:42.584369 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:42.584336    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 09 21:51:42.584758 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:42.584386    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg" podUID="23d09d41-47e5-4384-b6d1-df3b4087e544" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 09 21:51:43.443988 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.443944    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"]
Sep 09 21:51:43.448695 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.448673    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.450751 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.450731    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-9e275c-openshift-default-dockercfg-bzmqm\""
Sep 09 21:51:43.461121 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.461095    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"]
Sep 09 21:51:43.594863 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.594828    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:43.596367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.596343    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:43.597252 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.597228    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:51:43.628418 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.628393    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-data\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.628510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.628432    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-podinfo\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.628510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.628451    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-envoy\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.628510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.628473    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-workload-certs\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.628627 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.628531    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istiod-ca-cert\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.628627 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.628609    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ph8x\" (UniqueName: \"kubernetes.io/projected/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-kube-api-access-9ph8x\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.628695 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.628660    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-workload-socket\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.628729 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.628713    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-credential-socket\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.628763 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.628750    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-token\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.729925 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.729853    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ph8x\" (UniqueName: \"kubernetes.io/projected/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-kube-api-access-9ph8x\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.729925 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.729900    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-workload-socket\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.730149 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.729931    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-credential-socket\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.730149 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.729963    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-token\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.730149 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.730089    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-data\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.730149 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.730133    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-podinfo\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.730370 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.730162    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-envoy\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.730370 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.730293    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-workload-socket\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.730370 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.730342    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-workload-certs\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.730370 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.730345    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-credential-socket\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.730370 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.730370    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istiod-ca-cert\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.730653 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.730473    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-data\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.730713 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.730658    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-workload-certs\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.731084 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.731059    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istiod-ca-cert\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.732378 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.732356    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-envoy\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.732770 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.732751    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-podinfo\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.736829 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.736809    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-token\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.737063 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.737047    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ph8x\" (UniqueName: \"kubernetes.io/projected/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-kube-api-access-9ph8x\") pod \"e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") " pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:43.764782 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:43.762152    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:44.044389 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:44.044362    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"]
Sep 09 21:51:44.045387 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:51:44.045359    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd783cd0c_d8a3_44c7_b9fd_39c0821873fa.slice/crio-27f15a402fdefcfb945da265c1757eba576accbef8316a35cfea7b6e753c4014 WatchSource:0}: Error finding container 27f15a402fdefcfb945da265c1757eba576accbef8316a35cfea7b6e753c4014: Status 404 returned error can't find the container with id 27f15a402fdefcfb945da265c1757eba576accbef8316a35cfea7b6e753c4014
Sep 09 21:51:44.047900 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:44.047862    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:51:44.047990 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:44.047951    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:51:44.048039 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:44.048012    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:51:44.340341 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:44.340261    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4" event={"ID":"d783cd0c-d8a3-44c7-b9fd-39c0821873fa","Type":"ContainerStarted","Data":"8e44b2db8ab47322f61d60b8faff767f27de6882cbdfb88cde7851c1781c634e"}
Sep 09 21:51:44.340341 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:44.340317    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4" event={"ID":"d783cd0c-d8a3-44c7-b9fd-39c0821873fa","Type":"ContainerStarted","Data":"27f15a402fdefcfb945da265c1757eba576accbef8316a35cfea7b6e753c4014"}
Sep 09 21:51:44.360865 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:44.360815    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4" podStartSLOduration=1.360801226 podStartE2EDuration="1.360801226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:51:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:51:44.35956137 +0000 UTC m=+1123.832526281" watchObservedRunningTime="2026-09-09 21:51:44.360801226 +0000 UTC m=+1123.833766135"
Sep 09 21:51:44.763620 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:44.763549    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:45.768232 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:45.768201    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:46.352900 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:46.352860    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:46.353783 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:46.353764    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:51:52.204856 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:52.204823    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"]
Sep 09 21:51:55.769000 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:51:55.768953    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:52:06.493155 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:06.493100    2578 generic.go:356] "Generic (PLEG): container finished" podID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" containerID="8e44b2db8ab47322f61d60b8faff767f27de6882cbdfb88cde7851c1781c634e" exitCode=0
Sep 09 21:52:06.493155 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:06.493147    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4" event={"ID":"d783cd0c-d8a3-44c7-b9fd-39c0821873fa","Type":"ContainerDied","Data":"8e44b2db8ab47322f61d60b8faff767f27de6882cbdfb88cde7851c1781c634e"}
Sep 09 21:52:06.493553 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:06.493507    2578 scope.go:117] "RemoveContainer" containerID="8e44b2db8ab47322f61d60b8faff767f27de6882cbdfb88cde7851c1781c634e"
Sep 09 21:52:06.494677 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:06.494645    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:52:06.494776 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:06.494716    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:52:06.494776 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:06.494760    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:52:07.225380 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.225339    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx" podUID="0ce57c82-c33d-41c0-98ba-983ec35fa28f" containerName="main" containerID="cri-o://495071bf5629b75c930c46216d1fbd508d2e094428944a57c47b3f934d9f4b73" gracePeriod=45
Sep 09 21:52:07.501700 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.501671    2578 generic.go:356] "Generic (PLEG): container finished" podID="0ce57c82-c33d-41c0-98ba-983ec35fa28f" containerID="495071bf5629b75c930c46216d1fbd508d2e094428944a57c47b3f934d9f4b73" exitCode=0
Sep 09 21:52:07.502090 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.501755    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx" event={"ID":"0ce57c82-c33d-41c0-98ba-983ec35fa28f","Type":"ContainerDied","Data":"495071bf5629b75c930c46216d1fbd508d2e094428944a57c47b3f934d9f4b73"}
Sep 09 21:52:07.502090 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.501822    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx" event={"ID":"0ce57c82-c33d-41c0-98ba-983ec35fa28f","Type":"ContainerDied","Data":"dec21f31148c6fd87bc43a9dacbe77b3cfbf14e8aae83a94ceeef740917f95f9"}
Sep 09 21:52:07.502090 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.501839    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dec21f31148c6fd87bc43a9dacbe77b3cfbf14e8aae83a94ceeef740917f95f9"
Sep 09 21:52:07.504022 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.503997    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4" event={"ID":"d783cd0c-d8a3-44c7-b9fd-39c0821873fa","Type":"ContainerStarted","Data":"dd050cbb47a0a0cb32179d2c32aafd8ca6fceb5222fb17ba5e9486c8d18517ce"}
Sep 09 21:52:07.517371 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.517354    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:52:07.665334 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.665299    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-tmp-dir\") pod \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") "
Sep 09 21:52:07.665648 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.665625    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ce57c82-c33d-41c0-98ba-983ec35fa28f-tls-certs\") pod \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") "
Sep 09 21:52:07.666337 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.665733    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0ce57c82-c33d-41c0-98ba-983ec35fa28f" (UID: "0ce57c82-c33d-41c0-98ba-983ec35fa28f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:52:07.666337 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.665866    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-model-cache\") pod \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") "
Sep 09 21:52:07.666337 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.665905    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqsfk\" (UniqueName: \"kubernetes.io/projected/0ce57c82-c33d-41c0-98ba-983ec35fa28f-kube-api-access-gqsfk\") pod \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") "
Sep 09 21:52:07.666337 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.665983    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-home\") pod \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") "
Sep 09 21:52:07.666337 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.666011    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-dshm\") pod \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\" (UID: \"0ce57c82-c33d-41c0-98ba-983ec35fa28f\") "
Sep 09 21:52:07.666337 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.666238    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-home" (OuterVolumeSpecName: "home") pod "0ce57c82-c33d-41c0-98ba-983ec35fa28f" (UID: "0ce57c82-c33d-41c0-98ba-983ec35fa28f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:52:07.666782 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.666527    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-tmp-dir\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:07.666782 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.666548    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-home\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:07.666897 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.666790    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-model-cache" (OuterVolumeSpecName: "model-cache") pod "0ce57c82-c33d-41c0-98ba-983ec35fa28f" (UID: "0ce57c82-c33d-41c0-98ba-983ec35fa28f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:52:07.668659 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.668636    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-dshm" (OuterVolumeSpecName: "dshm") pod "0ce57c82-c33d-41c0-98ba-983ec35fa28f" (UID: "0ce57c82-c33d-41c0-98ba-983ec35fa28f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:52:07.668742 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.668678    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ce57c82-c33d-41c0-98ba-983ec35fa28f-kube-api-access-gqsfk" (OuterVolumeSpecName: "kube-api-access-gqsfk") pod "0ce57c82-c33d-41c0-98ba-983ec35fa28f" (UID: "0ce57c82-c33d-41c0-98ba-983ec35fa28f"). InnerVolumeSpecName "kube-api-access-gqsfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:52:07.668791 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.668744    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ce57c82-c33d-41c0-98ba-983ec35fa28f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0ce57c82-c33d-41c0-98ba-983ec35fa28f" (UID: "0ce57c82-c33d-41c0-98ba-983ec35fa28f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:52:07.763042 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.762919    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:52:07.763263 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.763244    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 09 21:52:07.763313 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.763288    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4" podUID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 09 21:52:07.767291 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.767270    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ce57c82-c33d-41c0-98ba-983ec35fa28f-tls-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:07.767291 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.767292    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-model-cache\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:07.767410 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.767301    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqsfk\" (UniqueName: \"kubernetes.io/projected/0ce57c82-c33d-41c0-98ba-983ec35fa28f-kube-api-access-gqsfk\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:07.767410 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:07.767312    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ce57c82-c33d-41c0-98ba-983ec35fa28f-dshm\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:08.518484 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:08.518452    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"
Sep 09 21:52:08.555324 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:08.555293    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"]
Sep 09 21:52:08.560402 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:08.560374    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-331c9d/explicit-model-331c9d-kserve-5b76d59f9f-26xdx"]
Sep 09 21:52:08.763809 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:08.763772    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 09 21:52:08.763960 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:08.763847    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4" podUID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 09 21:52:09.003436 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:09.003360    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ce57c82-c33d-41c0-98ba-983ec35fa28f" path="/var/lib/kubelet/pods/0ce57c82-c33d-41c0-98ba-983ec35fa28f/volumes"
Sep 09 21:52:09.763923 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:09.763884    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 09 21:52:09.764321 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:09.763991    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4" podUID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 09 21:52:10.763884 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:10.763851    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 09 21:52:10.764073 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:10.763914    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4" podUID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 09 21:52:11.763523 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:11.763489    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 09 21:52:11.763702 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:11.763546    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4" podUID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 09 21:52:11.920637 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:11.920605    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7492-b80dd6"]
Sep 09 21:52:11.921054 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:11.921039    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ce57c82-c33d-41c0-98ba-983ec35fa28f" containerName="main"
Sep 09 21:52:11.921110 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:11.921056    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ce57c82-c33d-41c0-98ba-983ec35fa28f" containerName="main"
Sep 09 21:52:11.921150 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:11.921135    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ce57c82-c33d-41c0-98ba-983ec35fa28f" containerName="main"
Sep 09 21:52:11.923902 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:11.923885    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-b80dd6"
Sep 09 21:52:11.926618 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:11.926597    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-k4l58\""
Sep 09 21:52:11.937141 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:11.937108    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7492-b80dd6"]
Sep 09 21:52:12.012241 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:12.012208    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkw27\" (UniqueName: \"kubernetes.io/projected/53f4b0f5-a485-44e4-b1af-842cff89c922-kube-api-access-fkw27\") pod \"test-curl-7492-b80dd6\" (UID: \"53f4b0f5-a485-44e4-b1af-842cff89c922\") " pod="opendatahub/test-curl-7492-b80dd6"
Sep 09 21:52:12.113806 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:12.113774    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkw27\" (UniqueName: \"kubernetes.io/projected/53f4b0f5-a485-44e4-b1af-842cff89c922-kube-api-access-fkw27\") pod \"test-curl-7492-b80dd6\" (UID: \"53f4b0f5-a485-44e4-b1af-842cff89c922\") " pod="opendatahub/test-curl-7492-b80dd6"
Sep 09 21:52:12.124872 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:12.124842    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkw27\" (UniqueName: \"kubernetes.io/projected/53f4b0f5-a485-44e4-b1af-842cff89c922-kube-api-access-fkw27\") pod \"test-curl-7492-b80dd6\" (UID: \"53f4b0f5-a485-44e4-b1af-842cff89c922\") " pod="opendatahub/test-curl-7492-b80dd6"
Sep 09 21:52:12.238626 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:12.238587    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-b80dd6"
Sep 09 21:52:12.520622 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:12.520598    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7492-b80dd6"]
Sep 09 21:52:12.522656 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:52:12.522628    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53f4b0f5_a485_44e4_b1af_842cff89c922.slice/crio-fb44253b4750340a570e1442df64a6d05ea45dc63b26b2dd960d4dff3d77c880 WatchSource:0}: Error finding container fb44253b4750340a570e1442df64a6d05ea45dc63b26b2dd960d4dff3d77c880: Status 404 returned error can't find the container with id fb44253b4750340a570e1442df64a6d05ea45dc63b26b2dd960d4dff3d77c880
Sep 09 21:52:12.558573 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:12.558541    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7492-b80dd6" event={"ID":"53f4b0f5-a485-44e4-b1af-842cff89c922","Type":"ContainerStarted","Data":"fb44253b4750340a570e1442df64a6d05ea45dc63b26b2dd960d4dff3d77c880"}
Sep 09 21:52:12.767723 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:12.767694    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:52:12.768001 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:12.767955    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:52:12.768527 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:12.768508    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 21:52:13.566255 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:13.566217    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7492-b80dd6" event={"ID":"53f4b0f5-a485-44e4-b1af-842cff89c922","Type":"ContainerStarted","Data":"13ffaab8ca7d471c078716e14117bc10322478f2d072036071ca930a6465c311"}
Sep 09 21:52:13.579640 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:13.579594    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7492-b80dd6" podStartSLOduration=2.579577288 podStartE2EDuration="2.579577288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:52:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:52:13.579345704 +0000 UTC m=+1153.052310614" watchObservedRunningTime="2026-09-09 21:52:13.579577288 +0000 UTC m=+1153.052542198"
Sep 09 21:52:14.137814 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:14.137781    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7492-b80dd6"]
Sep 09 21:52:14.140859 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:14.140832    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7492-b80dd6"]
Sep 09 21:52:14.249045 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:14.248964    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7492-6434aa"]
Sep 09 21:52:14.249413 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:14.249399    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53f4b0f5-a485-44e4-b1af-842cff89c922" containerName="test-curl-7492-b80dd6"
Sep 09 21:52:14.249459 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:14.249414    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="53f4b0f5-a485-44e4-b1af-842cff89c922" containerName="test-curl-7492-b80dd6"
Sep 09 21:52:14.249529 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:14.249502    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="53f4b0f5-a485-44e4-b1af-842cff89c922" containerName="test-curl-7492-b80dd6"
Sep 09 21:52:14.251744 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:14.251728    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-6434aa"
Sep 09 21:52:14.257937 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:14.257909    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7492-6434aa"]
Sep 09 21:52:14.338342 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:14.338312    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g25h5\" (UniqueName: \"kubernetes.io/projected/50e41cd8-9711-467a-a086-21fc3bb57ec0-kube-api-access-g25h5\") pod \"test-curl-7492-6434aa\" (UID: \"50e41cd8-9711-467a-a086-21fc3bb57ec0\") " pod="opendatahub/test-curl-7492-6434aa"
Sep 09 21:52:14.439557 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:14.439528    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g25h5\" (UniqueName: \"kubernetes.io/projected/50e41cd8-9711-467a-a086-21fc3bb57ec0-kube-api-access-g25h5\") pod \"test-curl-7492-6434aa\" (UID: \"50e41cd8-9711-467a-a086-21fc3bb57ec0\") " pod="opendatahub/test-curl-7492-6434aa"
Sep 09 21:52:14.446858 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:14.446820    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g25h5\" (UniqueName: \"kubernetes.io/projected/50e41cd8-9711-467a-a086-21fc3bb57ec0-kube-api-access-g25h5\") pod \"test-curl-7492-6434aa\" (UID: \"50e41cd8-9711-467a-a086-21fc3bb57ec0\") " pod="opendatahub/test-curl-7492-6434aa"
Sep 09 21:52:14.562073 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:14.562046    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-6434aa"
Sep 09 21:52:14.572347 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:14.572328    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7492-b80dd6" container="test-curl-7492-b80dd6"
Sep 09 21:52:14.828852 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:14.828830    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7492-6434aa"]
Sep 09 21:52:14.830217 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:52:14.830191    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50e41cd8_9711_467a_a086_21fc3bb57ec0.slice/crio-5c286cc0c11d55550626e8a4a14fe09c3a547867aa02e9df671e356e319992c5 WatchSource:0}: Error finding container 5c286cc0c11d55550626e8a4a14fe09c3a547867aa02e9df671e356e319992c5: Status 404 returned error can't find the container with id 5c286cc0c11d55550626e8a4a14fe09c3a547867aa02e9df671e356e319992c5
Sep 09 21:52:15.580039 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:15.580002    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7492-6434aa" event={"ID":"50e41cd8-9711-467a-a086-21fc3bb57ec0","Type":"ContainerStarted","Data":"6fedb85e03dab59e152eb0805284937973c1ff8b59cfbe2fafb6b3bcd334edfd"}
Sep 09 21:52:15.580541 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:15.580048    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7492-6434aa" event={"ID":"50e41cd8-9711-467a-a086-21fc3bb57ec0","Type":"ContainerStarted","Data":"5c286cc0c11d55550626e8a4a14fe09c3a547867aa02e9df671e356e319992c5"}
Sep 09 21:52:15.580541 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:15.580139    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7492-b80dd6" podUID="53f4b0f5-a485-44e4-b1af-842cff89c922" containerName="test-curl-7492-b80dd6" containerID="cri-o://13ffaab8ca7d471c078716e14117bc10322478f2d072036071ca930a6465c311" gracePeriod=2
Sep 09 21:52:15.595222 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:15.595179    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7492-6434aa" podStartSLOduration=1.595165333 podStartE2EDuration="1.595165333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:52:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:52:15.593910316 +0000 UTC m=+1155.066875229" watchObservedRunningTime="2026-09-09 21:52:15.595165333 +0000 UTC m=+1155.068130240"
Sep 09 21:52:16.171590 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:16.171557    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7492-6434aa"]
Sep 09 21:52:16.175614 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:16.175583    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7492-6434aa"]
Sep 09 21:52:16.284493 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:16.284462    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7492-dca84d"]
Sep 09 21:52:16.284889 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:16.284876    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50e41cd8-9711-467a-a086-21fc3bb57ec0" containerName="test-curl-7492-6434aa"
Sep 09 21:52:16.284942 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:16.284891    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="50e41cd8-9711-467a-a086-21fc3bb57ec0" containerName="test-curl-7492-6434aa"
Sep 09 21:52:16.285805 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:16.285777    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="50e41cd8-9711-467a-a086-21fc3bb57ec0" containerName="test-curl-7492-6434aa"
Sep 09 21:52:16.294702 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:16.294554    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-dca84d"
Sep 09 21:52:16.295708 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:16.295683    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7492-dca84d"]
Sep 09 21:52:16.460327 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:16.460298    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9j7j\" (UniqueName: \"kubernetes.io/projected/f35360c1-a448-40ca-94e1-7e4a0a2cb940-kube-api-access-k9j7j\") pod \"test-curl-7492-dca84d\" (UID: \"f35360c1-a448-40ca-94e1-7e4a0a2cb940\") " pod="opendatahub/test-curl-7492-dca84d"
Sep 09 21:52:16.561644 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:16.561572    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9j7j\" (UniqueName: \"kubernetes.io/projected/f35360c1-a448-40ca-94e1-7e4a0a2cb940-kube-api-access-k9j7j\") pod \"test-curl-7492-dca84d\" (UID: \"f35360c1-a448-40ca-94e1-7e4a0a2cb940\") " pod="opendatahub/test-curl-7492-dca84d"
Sep 09 21:52:16.569173 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:16.569150    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9j7j\" (UniqueName: \"kubernetes.io/projected/f35360c1-a448-40ca-94e1-7e4a0a2cb940-kube-api-access-k9j7j\") pod \"test-curl-7492-dca84d\" (UID: \"f35360c1-a448-40ca-94e1-7e4a0a2cb940\") " pod="opendatahub/test-curl-7492-dca84d"
Sep 09 21:52:16.586600 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:16.586577    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7492-6434aa" container="test-curl-7492-6434aa"
Sep 09 21:52:16.604826 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:16.604808    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-dca84d"
Sep 09 21:52:16.880385 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:16.880358    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7492-dca84d"]
Sep 09 21:52:16.881533 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:52:16.881488    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf35360c1_a448_40ca_94e1_7e4a0a2cb940.slice/crio-f61cf8858e7f0da802a5eb75bbbc4322e9a342922242c4be16aa4e89c1a7fa07 WatchSource:0}: Error finding container f61cf8858e7f0da802a5eb75bbbc4322e9a342922242c4be16aa4e89c1a7fa07: Status 404 returned error can't find the container with id f61cf8858e7f0da802a5eb75bbbc4322e9a342922242c4be16aa4e89c1a7fa07
Sep 09 21:52:17.594394 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:17.594368    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7492-dca84d" event={"ID":"f35360c1-a448-40ca-94e1-7e4a0a2cb940","Type":"ContainerStarted","Data":"af027cdf0b05ce24152246b9fc64f220fdc30ef746e55b0f8f7fe50de717ab33"}
Sep 09 21:52:17.594674 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:17.594409    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7492-dca84d" event={"ID":"f35360c1-a448-40ca-94e1-7e4a0a2cb940","Type":"ContainerStarted","Data":"f61cf8858e7f0da802a5eb75bbbc4322e9a342922242c4be16aa4e89c1a7fa07"}
Sep 09 21:52:17.594674 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:17.594467    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7492-6434aa" podUID="50e41cd8-9711-467a-a086-21fc3bb57ec0" containerName="test-curl-7492-6434aa" containerID="cri-o://6fedb85e03dab59e152eb0805284937973c1ff8b59cfbe2fafb6b3bcd334edfd" gracePeriod=2
Sep 09 21:52:17.614311 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:17.614271    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7492-dca84d" podStartSLOduration=1.614257616 podStartE2EDuration="1.614257616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:52:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:52:17.611400074 +0000 UTC m=+1157.084364982" watchObservedRunningTime="2026-09-09 21:52:17.614257616 +0000 UTC m=+1157.087222526"
Sep 09 21:52:17.764282 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:17.764262    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-b80dd6"
Sep 09 21:52:17.874322 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:17.874262    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkw27\" (UniqueName: \"kubernetes.io/projected/53f4b0f5-a485-44e4-b1af-842cff89c922-kube-api-access-fkw27\") pod \"53f4b0f5-a485-44e4-b1af-842cff89c922\" (UID: \"53f4b0f5-a485-44e4-b1af-842cff89c922\") "
Sep 09 21:52:17.877280 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:17.877251    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53f4b0f5-a485-44e4-b1af-842cff89c922-kube-api-access-fkw27" (OuterVolumeSpecName: "kube-api-access-fkw27") pod "53f4b0f5-a485-44e4-b1af-842cff89c922" (UID: "53f4b0f5-a485-44e4-b1af-842cff89c922"). InnerVolumeSpecName "kube-api-access-fkw27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:52:17.975779 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:17.975758    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkw27\" (UniqueName: \"kubernetes.io/projected/53f4b0f5-a485-44e4-b1af-842cff89c922-kube-api-access-fkw27\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:18.195833 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.195752    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7492-dca84d"]
Sep 09 21:52:18.215614 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.215589    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7492-dca84d"]
Sep 09 21:52:18.318661 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.318631    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7492-8c7117"]
Sep 09 21:52:18.319122 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.319107    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f35360c1-a448-40ca-94e1-7e4a0a2cb940" containerName="test-curl-7492-dca84d"
Sep 09 21:52:18.319175 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.319124    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f35360c1-a448-40ca-94e1-7e4a0a2cb940" containerName="test-curl-7492-dca84d"
Sep 09 21:52:18.319248 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.319218    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f35360c1-a448-40ca-94e1-7e4a0a2cb940" containerName="test-curl-7492-dca84d"
Sep 09 21:52:18.322516 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.322492    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-8c7117"
Sep 09 21:52:18.325898 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.325875    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7492-8c7117"]
Sep 09 21:52:18.482190 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.482107    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v66m\" (UniqueName: \"kubernetes.io/projected/a65874c8-1146-4495-b13b-0ae2c3b54f6a-kube-api-access-2v66m\") pod \"test-curl-7492-8c7117\" (UID: \"a65874c8-1146-4495-b13b-0ae2c3b54f6a\") " pod="opendatahub/test-curl-7492-8c7117"
Sep 09 21:52:18.583394 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.583364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v66m\" (UniqueName: \"kubernetes.io/projected/a65874c8-1146-4495-b13b-0ae2c3b54f6a-kube-api-access-2v66m\") pod \"test-curl-7492-8c7117\" (UID: \"a65874c8-1146-4495-b13b-0ae2c3b54f6a\") " pod="opendatahub/test-curl-7492-8c7117"
Sep 09 21:52:18.591570 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.591538    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v66m\" (UniqueName: \"kubernetes.io/projected/a65874c8-1146-4495-b13b-0ae2c3b54f6a-kube-api-access-2v66m\") pod \"test-curl-7492-8c7117\" (UID: \"a65874c8-1146-4495-b13b-0ae2c3b54f6a\") " pod="opendatahub/test-curl-7492-8c7117"
Sep 09 21:52:18.602615 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.602583    2578 generic.go:356] "Generic (PLEG): container finished" podID="53f4b0f5-a485-44e4-b1af-842cff89c922" containerID="13ffaab8ca7d471c078716e14117bc10322478f2d072036071ca930a6465c311" exitCode=137
Sep 09 21:52:18.602954 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.602624    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-b80dd6"
Sep 09 21:52:18.602954 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.602671    2578 scope.go:117] "RemoveContainer" containerID="13ffaab8ca7d471c078716e14117bc10322478f2d072036071ca930a6465c311"
Sep 09 21:52:18.603089 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:18.603075    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7492-dca84d" container="test-curl-7492-dca84d"
Sep 09 21:52:18.612934 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.612918    2578 scope.go:117] "RemoveContainer" containerID="13ffaab8ca7d471c078716e14117bc10322478f2d072036071ca930a6465c311"
Sep 09 21:52:18.613215 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:18.613198    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13ffaab8ca7d471c078716e14117bc10322478f2d072036071ca930a6465c311\": container with ID starting with 13ffaab8ca7d471c078716e14117bc10322478f2d072036071ca930a6465c311 not found: ID does not exist" containerID="13ffaab8ca7d471c078716e14117bc10322478f2d072036071ca930a6465c311"
Sep 09 21:52:18.613272 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.613224    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13ffaab8ca7d471c078716e14117bc10322478f2d072036071ca930a6465c311"} err="failed to get container status \"13ffaab8ca7d471c078716e14117bc10322478f2d072036071ca930a6465c311\": rpc error: code = NotFound desc = could not find container \"13ffaab8ca7d471c078716e14117bc10322478f2d072036071ca930a6465c311\": container with ID starting with 13ffaab8ca7d471c078716e14117bc10322478f2d072036071ca930a6465c311 not found: ID does not exist"
Sep 09 21:52:18.613763 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:18.613741    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7492-b80dd6\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="53f4b0f5-a485-44e4-b1af-842cff89c922" pod="opendatahub/test-curl-7492-b80dd6"
Sep 09 21:52:18.632950 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.632930    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-8c7117"
Sep 09 21:52:18.907408 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:18.907385    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7492-8c7117"]
Sep 09 21:52:18.908749 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:52:18.908724    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda65874c8_1146_4495_b13b_0ae2c3b54f6a.slice/crio-8356031b4d5f7edbb620a25a0b45cb3a854b395b7c2a78ce788cf59ffbfd2836 WatchSource:0}: Error finding container 8356031b4d5f7edbb620a25a0b45cb3a854b395b7c2a78ce788cf59ffbfd2836: Status 404 returned error can't find the container with id 8356031b4d5f7edbb620a25a0b45cb3a854b395b7c2a78ce788cf59ffbfd2836
Sep 09 21:52:19.003032 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:19.003007    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53f4b0f5-a485-44e4-b1af-842cff89c922" path="/var/lib/kubelet/pods/53f4b0f5-a485-44e4-b1af-842cff89c922/volumes"
Sep 09 21:52:19.611341 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:19.611310    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7492-8c7117" event={"ID":"a65874c8-1146-4495-b13b-0ae2c3b54f6a","Type":"ContainerStarted","Data":"2654dc630a2ec5b615a8c3bd1d5216202c7f1a31f95ddc7403ec4b5d8375747e"}
Sep 09 21:52:19.611681 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:19.611359    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7492-8c7117" event={"ID":"a65874c8-1146-4495-b13b-0ae2c3b54f6a","Type":"ContainerStarted","Data":"8356031b4d5f7edbb620a25a0b45cb3a854b395b7c2a78ce788cf59ffbfd2836"}
Sep 09 21:52:19.612460 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:19.612436    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7492-dca84d" podUID="f35360c1-a448-40ca-94e1-7e4a0a2cb940" containerName="test-curl-7492-dca84d" containerID="cri-o://af027cdf0b05ce24152246b9fc64f220fdc30ef746e55b0f8f7fe50de717ab33" gracePeriod=2
Sep 09 21:52:19.635567 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:19.635513    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7492-8c7117" podStartSLOduration=1.635498049 podStartE2EDuration="1.635498049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:52:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:52:19.62573979 +0000 UTC m=+1159.098704699" watchObservedRunningTime="2026-09-09 21:52:19.635498049 +0000 UTC m=+1159.108462959"
Sep 09 21:52:19.779893 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:19.779873    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-6434aa"
Sep 09 21:52:19.895697 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:19.895631    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g25h5\" (UniqueName: \"kubernetes.io/projected/50e41cd8-9711-467a-a086-21fc3bb57ec0-kube-api-access-g25h5\") pod \"50e41cd8-9711-467a-a086-21fc3bb57ec0\" (UID: \"50e41cd8-9711-467a-a086-21fc3bb57ec0\") "
Sep 09 21:52:19.897588 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:19.897535    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50e41cd8-9711-467a-a086-21fc3bb57ec0-kube-api-access-g25h5" (OuterVolumeSpecName: "kube-api-access-g25h5") pod "50e41cd8-9711-467a-a086-21fc3bb57ec0" (UID: "50e41cd8-9711-467a-a086-21fc3bb57ec0"). InnerVolumeSpecName "kube-api-access-g25h5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:52:20.001411 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:19.999568    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g25h5\" (UniqueName: \"kubernetes.io/projected/50e41cd8-9711-467a-a086-21fc3bb57ec0-kube-api-access-g25h5\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:20.202825 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:20.202754    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7492-8c7117"]
Sep 09 21:52:20.207188 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:20.207164    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7492-8c7117"]
Sep 09 21:52:20.620703 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:20.620667    2578 generic.go:356] "Generic (PLEG): container finished" podID="50e41cd8-9711-467a-a086-21fc3bb57ec0" containerID="6fedb85e03dab59e152eb0805284937973c1ff8b59cfbe2fafb6b3bcd334edfd" exitCode=137
Sep 09 21:52:20.621186 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:20.620720    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-6434aa"
Sep 09 21:52:20.621186 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:20.620777    2578 scope.go:117] "RemoveContainer" containerID="6fedb85e03dab59e152eb0805284937973c1ff8b59cfbe2fafb6b3bcd334edfd"
Sep 09 21:52:20.621317 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:20.621213    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7492-8c7117" container="test-curl-7492-8c7117"
Sep 09 21:52:20.632704 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:20.632252    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7492-6434aa\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="50e41cd8-9711-467a-a086-21fc3bb57ec0" pod="opendatahub/test-curl-7492-6434aa"
Sep 09 21:52:20.634450 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:20.634429    2578 scope.go:117] "RemoveContainer" containerID="6fedb85e03dab59e152eb0805284937973c1ff8b59cfbe2fafb6b3bcd334edfd"
Sep 09 21:52:20.634749 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:20.634720    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fedb85e03dab59e152eb0805284937973c1ff8b59cfbe2fafb6b3bcd334edfd\": container with ID starting with 6fedb85e03dab59e152eb0805284937973c1ff8b59cfbe2fafb6b3bcd334edfd not found: ID does not exist" containerID="6fedb85e03dab59e152eb0805284937973c1ff8b59cfbe2fafb6b3bcd334edfd"
Sep 09 21:52:20.634835 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:20.634760    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fedb85e03dab59e152eb0805284937973c1ff8b59cfbe2fafb6b3bcd334edfd"} err="failed to get container status \"6fedb85e03dab59e152eb0805284937973c1ff8b59cfbe2fafb6b3bcd334edfd\": rpc error: code = NotFound desc = could not find container \"6fedb85e03dab59e152eb0805284937973c1ff8b59cfbe2fafb6b3bcd334edfd\": container with ID starting with 6fedb85e03dab59e152eb0805284937973c1ff8b59cfbe2fafb6b3bcd334edfd not found: ID does not exist"
Sep 09 21:52:21.002666 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:21.002573    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7492-6434aa\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="50e41cd8-9711-467a-a086-21fc3bb57ec0" pod="opendatahub/test-curl-7492-6434aa"
Sep 09 21:52:21.006526 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.006496    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50e41cd8-9711-467a-a086-21fc3bb57ec0" path="/var/lib/kubelet/pods/50e41cd8-9711-467a-a086-21fc3bb57ec0/volumes"
Sep 09 21:52:21.022135 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.022102    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"]
Sep 09 21:52:21.022589 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.022572    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a65874c8-1146-4495-b13b-0ae2c3b54f6a" containerName="test-curl-7492-8c7117"
Sep 09 21:52:21.022676 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.022593    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65874c8-1146-4495-b13b-0ae2c3b54f6a" containerName="test-curl-7492-8c7117"
Sep 09 21:52:21.022756 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.022742    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a65874c8-1146-4495-b13b-0ae2c3b54f6a" containerName="test-curl-7492-8c7117"
Sep 09 21:52:21.027259 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.027240    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.029371 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.029354    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4883373b-openshift-default-dockercfg-6wsvg\""
Sep 09 21:52:21.036460 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.036438    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"]
Sep 09 21:52:21.114002 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.111703    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-workload-certs\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.114002 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.111760    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wskfn\" (UniqueName: \"kubernetes.io/projected/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-kube-api-access-wskfn\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.114002 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.111794    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istiod-ca-cert\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.114002 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.111856    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-data\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.114002 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.111882    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-podinfo\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.114002 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.111941    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-workload-socket\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.114002 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.112008    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-envoy\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.114002 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.112118    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-credential-socket\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.114002 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.112149    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-token\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.213142 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.213110    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-data\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.213142 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.213144    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-podinfo\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.213356 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.213184    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-workload-socket\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.213356 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.213204    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-envoy\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.213356 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.213290    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-credential-socket\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.213356 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.213321    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-token\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.213609 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.213383    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-workload-certs\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.213609 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.213410    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wskfn\" (UniqueName: \"kubernetes.io/projected/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-kube-api-access-wskfn\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.213609 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.213438    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istiod-ca-cert\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.213775 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.213607    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-data\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.213775 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.213626    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-workload-socket\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.213775 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.213719    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-credential-socket\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.213926 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.213820    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-workload-certs\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.214100 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.214080    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istiod-ca-cert\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.215751 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.215730    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-podinfo\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.215863 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.215748    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-envoy\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.220743 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.220720    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-token\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.220946 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.220929    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wskfn\" (UniqueName: \"kubernetes.io/projected/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-kube-api-access-wskfn\") pod \"e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") " pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.340212 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.340191    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:21.625933 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.625907    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"]
Sep 09 21:52:21.626267 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:52:21.626128    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e09b6ed_f686_4bf8_9aa0_22859f5eed27.slice/crio-0230f1a20f985e03966990179e2f3350fac5951f035358942c7f0690ea4f94d6 WatchSource:0}: Error finding container 0230f1a20f985e03966990179e2f3350fac5951f035358942c7f0690ea4f94d6: Status 404 returned error can't find the container with id 0230f1a20f985e03966990179e2f3350fac5951f035358942c7f0690ea4f94d6
Sep 09 21:52:21.633108 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.633073    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7492-8c7117" podUID="a65874c8-1146-4495-b13b-0ae2c3b54f6a" containerName="test-curl-7492-8c7117" containerID="cri-o://2654dc630a2ec5b615a8c3bd1d5216202c7f1a31f95ddc7403ec4b5d8375747e" gracePeriod=2
Sep 09 21:52:21.648602 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.648572    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:52:21.648683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.648637    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:52:21.648683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.648669    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:52:21.784772 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.784743    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-dca84d"
Sep 09 21:52:21.924895 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.924813    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9j7j\" (UniqueName: \"kubernetes.io/projected/f35360c1-a448-40ca-94e1-7e4a0a2cb940-kube-api-access-k9j7j\") pod \"f35360c1-a448-40ca-94e1-7e4a0a2cb940\" (UID: \"f35360c1-a448-40ca-94e1-7e4a0a2cb940\") "
Sep 09 21:52:21.926948 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:21.926926    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f35360c1-a448-40ca-94e1-7e4a0a2cb940-kube-api-access-k9j7j" (OuterVolumeSpecName: "kube-api-access-k9j7j") pod "f35360c1-a448-40ca-94e1-7e4a0a2cb940" (UID: "f35360c1-a448-40ca-94e1-7e4a0a2cb940"). InnerVolumeSpecName "kube-api-access-k9j7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:52:22.026328 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:22.026298    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9j7j\" (UniqueName: \"kubernetes.io/projected/f35360c1-a448-40ca-94e1-7e4a0a2cb940-kube-api-access-k9j7j\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:22.641099 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:22.641072    2578 generic.go:356] "Generic (PLEG): container finished" podID="f35360c1-a448-40ca-94e1-7e4a0a2cb940" containerID="af027cdf0b05ce24152246b9fc64f220fdc30ef746e55b0f8f7fe50de717ab33" exitCode=137
Sep 09 21:52:22.641573 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:22.641111    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-dca84d"
Sep 09 21:52:22.641573 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:22.641173    2578 scope.go:117] "RemoveContainer" containerID="af027cdf0b05ce24152246b9fc64f220fdc30ef746e55b0f8f7fe50de717ab33"
Sep 09 21:52:22.643026 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:22.642998    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r" event={"ID":"4e09b6ed-f686-4bf8-9aa0-22859f5eed27","Type":"ContainerStarted","Data":"38ef4080a7abc67761a9e9df95a8691596361073041afcf8604483b169c8f5b9"}
Sep 09 21:52:22.643132 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:22.643057    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r" event={"ID":"4e09b6ed-f686-4bf8-9aa0-22859f5eed27","Type":"ContainerStarted","Data":"0230f1a20f985e03966990179e2f3350fac5951f035358942c7f0690ea4f94d6"}
Sep 09 21:52:22.650859 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:22.650844    2578 scope.go:117] "RemoveContainer" containerID="af027cdf0b05ce24152246b9fc64f220fdc30ef746e55b0f8f7fe50de717ab33"
Sep 09 21:52:22.651139 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:22.651122    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af027cdf0b05ce24152246b9fc64f220fdc30ef746e55b0f8f7fe50de717ab33\": container with ID starting with af027cdf0b05ce24152246b9fc64f220fdc30ef746e55b0f8f7fe50de717ab33 not found: ID does not exist" containerID="af027cdf0b05ce24152246b9fc64f220fdc30ef746e55b0f8f7fe50de717ab33"
Sep 09 21:52:22.651179 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:22.651148    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af027cdf0b05ce24152246b9fc64f220fdc30ef746e55b0f8f7fe50de717ab33"} err="failed to get container status \"af027cdf0b05ce24152246b9fc64f220fdc30ef746e55b0f8f7fe50de717ab33\": rpc error: code = NotFound desc = could not find container \"af027cdf0b05ce24152246b9fc64f220fdc30ef746e55b0f8f7fe50de717ab33\": container with ID starting with af027cdf0b05ce24152246b9fc64f220fdc30ef746e55b0f8f7fe50de717ab33 not found: ID does not exist"
Sep 09 21:52:22.662082 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:22.662044    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r" podStartSLOduration=2.662031323 podStartE2EDuration="2.662031323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:52:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:52:22.660284669 +0000 UTC m=+1162.133249570" watchObservedRunningTime="2026-09-09 21:52:22.662031323 +0000 UTC m=+1162.134996229"
Sep 09 21:52:22.662179 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:22.662125    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7492-dca84d\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="f35360c1-a448-40ca-94e1-7e4a0a2cb940" pod="opendatahub/test-curl-7492-dca84d"
Sep 09 21:52:23.006321 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:23.006239    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f35360c1-a448-40ca-94e1-7e4a0a2cb940" path="/var/lib/kubelet/pods/f35360c1-a448-40ca-94e1-7e4a0a2cb940/volumes"
Sep 09 21:52:23.341239 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:23.341211    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:23.346660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:23.346632    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:23.651452 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:23.651429    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:23.652247 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:23.652228    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:52:23.825766 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:23.825742    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-8c7117"
Sep 09 21:52:23.943243 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:23.943161    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2v66m\" (UniqueName: \"kubernetes.io/projected/a65874c8-1146-4495-b13b-0ae2c3b54f6a-kube-api-access-2v66m\") pod \"a65874c8-1146-4495-b13b-0ae2c3b54f6a\" (UID: \"a65874c8-1146-4495-b13b-0ae2c3b54f6a\") "
Sep 09 21:52:23.945175 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:23.945146    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a65874c8-1146-4495-b13b-0ae2c3b54f6a-kube-api-access-2v66m" (OuterVolumeSpecName: "kube-api-access-2v66m") pod "a65874c8-1146-4495-b13b-0ae2c3b54f6a" (UID: "a65874c8-1146-4495-b13b-0ae2c3b54f6a"). InnerVolumeSpecName "kube-api-access-2v66m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:52:24.044421 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:24.044395    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2v66m\" (UniqueName: \"kubernetes.io/projected/a65874c8-1146-4495-b13b-0ae2c3b54f6a-kube-api-access-2v66m\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:24.658851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:24.658814    2578 generic.go:356] "Generic (PLEG): container finished" podID="a65874c8-1146-4495-b13b-0ae2c3b54f6a" containerID="2654dc630a2ec5b615a8c3bd1d5216202c7f1a31f95ddc7403ec4b5d8375747e" exitCode=137
Sep 09 21:52:24.658851 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:24.658849    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7492-8c7117"
Sep 09 21:52:24.659406 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:24.658901    2578 scope.go:117] "RemoveContainer" containerID="2654dc630a2ec5b615a8c3bd1d5216202c7f1a31f95ddc7403ec4b5d8375747e"
Sep 09 21:52:24.668315 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:24.668298    2578 scope.go:117] "RemoveContainer" containerID="2654dc630a2ec5b615a8c3bd1d5216202c7f1a31f95ddc7403ec4b5d8375747e"
Sep 09 21:52:24.668567 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:24.668548    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2654dc630a2ec5b615a8c3bd1d5216202c7f1a31f95ddc7403ec4b5d8375747e\": container with ID starting with 2654dc630a2ec5b615a8c3bd1d5216202c7f1a31f95ddc7403ec4b5d8375747e not found: ID does not exist" containerID="2654dc630a2ec5b615a8c3bd1d5216202c7f1a31f95ddc7403ec4b5d8375747e"
Sep 09 21:52:24.668641 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:24.668573    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2654dc630a2ec5b615a8c3bd1d5216202c7f1a31f95ddc7403ec4b5d8375747e"} err="failed to get container status \"2654dc630a2ec5b615a8c3bd1d5216202c7f1a31f95ddc7403ec4b5d8375747e\": rpc error: code = NotFound desc = could not find container \"2654dc630a2ec5b615a8c3bd1d5216202c7f1a31f95ddc7403ec4b5d8375747e\": container with ID starting with 2654dc630a2ec5b615a8c3bd1d5216202c7f1a31f95ddc7403ec4b5d8375747e not found: ID does not exist"
Sep 09 21:52:24.669047 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:24.669025    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7492-8c7117\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="a65874c8-1146-4495-b13b-0ae2c3b54f6a" pod="opendatahub/test-curl-7492-8c7117"
Sep 09 21:52:25.003846 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.003767    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a65874c8-1146-4495-b13b-0ae2c3b54f6a" path="/var/lib/kubelet/pods/a65874c8-1146-4495-b13b-0ae2c3b54f6a/volumes"
Sep 09 21:52:25.618624 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.618592    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"]
Sep 09 21:52:25.623777 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.623756    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:25.625850 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.625831    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4883373b-serving-cert\""
Sep 09 21:52:25.631083 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.631062    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"]
Sep 09 21:52:25.764168 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.764126    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9a6d61e-022e-4117-8299-53981a03c554-maas-api-tls\") pod \"maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:25.764701 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.764246    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a6d61e-022e-4117-8299-53981a03c554-openshift-service-ca\") pod \"maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:25.764701 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.764403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9a6d61e-022e-4117-8299-53981a03c554-metrics-certs\") pod \"maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:25.764701 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.764429    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlvmz\" (UniqueName: \"kubernetes.io/projected/d9a6d61e-022e-4117-8299-53981a03c554-kube-api-access-tlvmz\") pod \"maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:25.865257 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.865226    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9a6d61e-022e-4117-8299-53981a03c554-maas-api-tls\") pod \"maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:25.865448 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.865292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a6d61e-022e-4117-8299-53981a03c554-openshift-service-ca\") pod \"maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:25.865448 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.865353    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9a6d61e-022e-4117-8299-53981a03c554-metrics-certs\") pod \"maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:25.865448 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:25.865377    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-serving-cert: secret "maas-api-e2e-mt-4883373b-serving-cert" not found
Sep 09 21:52:25.865448 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.865392    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlvmz\" (UniqueName: \"kubernetes.io/projected/d9a6d61e-022e-4117-8299-53981a03c554-kube-api-access-tlvmz\") pod \"maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:25.865448 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:25.865438    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9a6d61e-022e-4117-8299-53981a03c554-maas-api-tls podName:d9a6d61e-022e-4117-8299-53981a03c554 nodeName:}" failed. No retries permitted until 2026-09-09 21:52:26.365421557 +0000 UTC m=+1165.838386445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d9a6d61e-022e-4117-8299-53981a03c554-maas-api-tls") pod "maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j" (UID: "d9a6d61e-022e-4117-8299-53981a03c554") : secret "maas-api-e2e-mt-4883373b-serving-cert" not found
Sep 09 21:52:25.865956 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.865936    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a6d61e-022e-4117-8299-53981a03c554-openshift-service-ca\") pod \"maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:25.867839 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.867815    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9a6d61e-022e-4117-8299-53981a03c554-metrics-certs\") pod \"maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:25.873815 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:25.873764    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlvmz\" (UniqueName: \"kubernetes.io/projected/d9a6d61e-022e-4117-8299-53981a03c554-kube-api-access-tlvmz\") pod \"maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:26.370906 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:26.370858    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9a6d61e-022e-4117-8299-53981a03c554-maas-api-tls\") pod \"maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:26.386946 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:26.386906    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9a6d61e-022e-4117-8299-53981a03c554-maas-api-tls\") pod \"maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:26.536335 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:26.536307    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:26.807696 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:26.807672    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"]
Sep 09 21:52:26.809146 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:52:26.809121    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a6d61e_022e_4117_8299_53981a03c554.slice/crio-06b25023d35566ef28731cd54ec2d1daaa3b90037521b47918d403eadeaa5d53 WatchSource:0}: Error finding container 06b25023d35566ef28731cd54ec2d1daaa3b90037521b47918d403eadeaa5d53: Status 404 returned error can't find the container with id 06b25023d35566ef28731cd54ec2d1daaa3b90037521b47918d403eadeaa5d53
Sep 09 21:52:27.684150 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:27.684061    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j" event={"ID":"d9a6d61e-022e-4117-8299-53981a03c554","Type":"ContainerStarted","Data":"021080e57bc15a1d60748a210c6b77ddbc910fb79cb031784eb8c76f605b7bcb"}
Sep 09 21:52:27.684150 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:27.684124    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j" event={"ID":"d9a6d61e-022e-4117-8299-53981a03c554","Type":"ContainerStarted","Data":"06b25023d35566ef28731cd54ec2d1daaa3b90037521b47918d403eadeaa5d53"}
Sep 09 21:52:27.684320 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:27.684151    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:27.700162 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:27.700108    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j" podStartSLOduration=2.2095544560000002 podStartE2EDuration="2.700089478s" totalImagesPullingTime="490.535022ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:52:27.699161482 +0000 UTC m=+1167.172126402" watchObservedRunningTime="2026-09-09 21:52:27.700089478 +0000 UTC m=+1167.173054589"
Sep 09 21:52:30.969308 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:30.969272    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:52:33.696131 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:33.696100    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:52:52.191603 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:52.191560    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"]
Sep 09 21:52:52.192134 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:52.191839    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9" podUID="c99ff219-f707-4de1-8299-c7ea5bf1d199" containerName="istio-proxy" containerID="cri-o://d16cf1f6c8c15520acc05dfcb622f7cb0f14399c73055886cc190b30ae3448e9" gracePeriod=30
Sep 09 21:52:52.924161 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:52.924129    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"]
Sep 09 21:52:52.929154 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:52.929136    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:52.932109 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:52.932088    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-749a53-openshift-default-dockercfg-2g9fx\""
Sep 09 21:52:52.941874 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:52.941852    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"]
Sep 09 21:52:53.039992 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.036513    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/099567be-4032-48ea-b06a-eed206dc7028-istio-podinfo\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.039992 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.036566    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/099567be-4032-48ea-b06a-eed206dc7028-istio-token\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.039992 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.036596    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-istio-data\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.039992 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.036655    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/099567be-4032-48ea-b06a-eed206dc7028-istiod-ca-cert\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.039992 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.036682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-credential-socket\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.039992 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.036727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6fnz\" (UniqueName: \"kubernetes.io/projected/099567be-4032-48ea-b06a-eed206dc7028-kube-api-access-c6fnz\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.039992 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.036789    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-istio-envoy\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.039992 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.036830    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-workload-socket\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.039992 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.036862    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-workload-certs\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.138257 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.138192    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6fnz\" (UniqueName: \"kubernetes.io/projected/099567be-4032-48ea-b06a-eed206dc7028-kube-api-access-c6fnz\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.138710 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.138685    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-istio-envoy\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.138885 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.138871    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-workload-socket\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.139034 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.139018    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-workload-certs\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.139248 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.139234    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/099567be-4032-48ea-b06a-eed206dc7028-istio-podinfo\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.139374 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.139361    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/099567be-4032-48ea-b06a-eed206dc7028-istio-token\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.139482 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.139455    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-workload-socket\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.139630 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.139488    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-istio-data\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.139802 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.139782    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/099567be-4032-48ea-b06a-eed206dc7028-istiod-ca-cert\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.139923 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.139907    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-credential-socket\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.140400 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.140381    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-credential-socket\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.140509 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.139806    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-istio-data\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.140597 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.140481    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/099567be-4032-48ea-b06a-eed206dc7028-istiod-ca-cert\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.141802 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.140895    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-workload-certs\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.141802 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.141553    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-istio-envoy\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.153197 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.142507    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/099567be-4032-48ea-b06a-eed206dc7028-istio-podinfo\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.153197 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.150661    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6fnz\" (UniqueName: \"kubernetes.io/projected/099567be-4032-48ea-b06a-eed206dc7028-kube-api-access-c6fnz\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.155444 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.155417    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/099567be-4032-48ea-b06a-eed206dc7028-istio-token\") pod \"e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") " pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.241163 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.241074    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:53.532671 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.532637    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"]
Sep 09 21:52:53.544136 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:52:53.544101    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099567be_4032_48ea_b06a_eed206dc7028.slice/crio-92c165c8e0df01630faf2b56319b4dda8781f6217c25d7093dfe42d0046eea74 WatchSource:0}: Error finding container 92c165c8e0df01630faf2b56319b4dda8781f6217c25d7093dfe42d0046eea74: Status 404 returned error can't find the container with id 92c165c8e0df01630faf2b56319b4dda8781f6217c25d7093dfe42d0046eea74
Sep 09 21:52:53.547670 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.547627    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:52:53.548152 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.548124    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:52:53.548319 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.548299    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:52:53.854332 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.854224    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq" event={"ID":"099567be-4032-48ea-b06a-eed206dc7028","Type":"ContainerStarted","Data":"60d7aa3e4f850b5b6f73c5f43e1990896b4c437fa839eedc5407a9cde1cef2b1"}
Sep 09 21:52:53.854332 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.854301    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq" event={"ID":"099567be-4032-48ea-b06a-eed206dc7028","Type":"ContainerStarted","Data":"92c165c8e0df01630faf2b56319b4dda8781f6217c25d7093dfe42d0046eea74"}
Sep 09 21:52:53.879934 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:53.879881    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq" podStartSLOduration=1.879859473 podStartE2EDuration="1.879859473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:52:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:52:53.87943107 +0000 UTC m=+1193.352395996" watchObservedRunningTime="2026-09-09 21:52:53.879859473 +0000 UTC m=+1193.352824386"
Sep 09 21:52:54.242005 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:54.241889    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:54.243819 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:54.243784    2578 patch_prober.go:28] interesting pod/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 09 21:52:54.243956 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:54.243843    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq" podUID="099567be-4032-48ea-b06a-eed206dc7028" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 09 21:52:55.257024 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:55.256994    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:55.873444 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:55.873415    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:55.876251 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:55.876221    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:52:57.488775 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.488753    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:52:57.499654 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.499635    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-workload-certs\") pod \"c99ff219-f707-4de1-8299-c7ea5bf1d199\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") "
Sep 09 21:52:57.499746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.499665    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-podinfo\") pod \"c99ff219-f707-4de1-8299-c7ea5bf1d199\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") "
Sep 09 21:52:57.499746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.499682    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-workload-socket\") pod \"c99ff219-f707-4de1-8299-c7ea5bf1d199\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") "
Sep 09 21:52:57.499746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.499736    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-data\") pod \"c99ff219-f707-4de1-8299-c7ea5bf1d199\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") "
Sep 09 21:52:57.499911 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.499754    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-envoy\") pod \"c99ff219-f707-4de1-8299-c7ea5bf1d199\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") "
Sep 09 21:52:57.499911 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.499773    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c99ff219-f707-4de1-8299-c7ea5bf1d199-istiod-ca-cert\") pod \"c99ff219-f707-4de1-8299-c7ea5bf1d199\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") "
Sep 09 21:52:57.499911 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.499792    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-token\") pod \"c99ff219-f707-4de1-8299-c7ea5bf1d199\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") "
Sep 09 21:52:57.499911 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.499837    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c99ff219-f707-4de1-8299-c7ea5bf1d199" (UID: "c99ff219-f707-4de1-8299-c7ea5bf1d199"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:52:57.500150 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.500019    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-data" (OuterVolumeSpecName: "istio-data") pod "c99ff219-f707-4de1-8299-c7ea5bf1d199" (UID: "c99ff219-f707-4de1-8299-c7ea5bf1d199"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:52:57.500150 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.500038    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-credential-socket\") pod \"c99ff219-f707-4de1-8299-c7ea5bf1d199\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") "
Sep 09 21:52:57.500150 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.500068    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c99ff219-f707-4de1-8299-c7ea5bf1d199" (UID: "c99ff219-f707-4de1-8299-c7ea5bf1d199"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:52:57.500150 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.500109    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kh8j\" (UniqueName: \"kubernetes.io/projected/c99ff219-f707-4de1-8299-c7ea5bf1d199-kube-api-access-4kh8j\") pod \"c99ff219-f707-4de1-8299-c7ea5bf1d199\" (UID: \"c99ff219-f707-4de1-8299-c7ea5bf1d199\") "
Sep 09 21:52:57.500150 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.500129    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c99ff219-f707-4de1-8299-c7ea5bf1d199-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c99ff219-f707-4de1-8299-c7ea5bf1d199" (UID: "c99ff219-f707-4de1-8299-c7ea5bf1d199"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:52:57.500410 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.500236    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c99ff219-f707-4de1-8299-c7ea5bf1d199" (UID: "c99ff219-f707-4de1-8299-c7ea5bf1d199"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:52:57.500549 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.500528    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:57.500635 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.500555    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:57.500635 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.500568    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c99ff219-f707-4de1-8299-c7ea5bf1d199-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:57.500635 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.500580    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:57.500635 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.500595    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:57.501929 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.501895    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c99ff219-f707-4de1-8299-c7ea5bf1d199" (UID: "c99ff219-f707-4de1-8299-c7ea5bf1d199"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:52:57.502106 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.502080    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c99ff219-f707-4de1-8299-c7ea5bf1d199" (UID: "c99ff219-f707-4de1-8299-c7ea5bf1d199"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:52:57.502221 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.502196    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c99ff219-f707-4de1-8299-c7ea5bf1d199-kube-api-access-4kh8j" (OuterVolumeSpecName: "kube-api-access-4kh8j") pod "c99ff219-f707-4de1-8299-c7ea5bf1d199" (UID: "c99ff219-f707-4de1-8299-c7ea5bf1d199"). InnerVolumeSpecName "kube-api-access-4kh8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:52:57.502262 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.502212    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-token" (OuterVolumeSpecName: "istio-token") pod "c99ff219-f707-4de1-8299-c7ea5bf1d199" (UID: "c99ff219-f707-4de1-8299-c7ea5bf1d199"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:52:57.601440 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.601413    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:57.601440 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.601437    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:57.601666 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.601446    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kh8j\" (UniqueName: \"kubernetes.io/projected/c99ff219-f707-4de1-8299-c7ea5bf1d199-kube-api-access-4kh8j\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:57.601666 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.601456    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c99ff219-f707-4de1-8299-c7ea5bf1d199-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:52:57.884870 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.884837    2578 generic.go:356] "Generic (PLEG): container finished" podID="c99ff219-f707-4de1-8299-c7ea5bf1d199" containerID="d16cf1f6c8c15520acc05dfcb622f7cb0f14399c73055886cc190b30ae3448e9" exitCode=0
Sep 09 21:52:57.885059 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.884894    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"
Sep 09 21:52:57.885059 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.884921    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9" event={"ID":"c99ff219-f707-4de1-8299-c7ea5bf1d199","Type":"ContainerDied","Data":"d16cf1f6c8c15520acc05dfcb622f7cb0f14399c73055886cc190b30ae3448e9"}
Sep 09 21:52:57.885059 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.885002    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9" event={"ID":"c99ff219-f707-4de1-8299-c7ea5bf1d199","Type":"ContainerDied","Data":"b5f1025e22b3b724bd513f940d02b804a5ea23dbf783396123480b9d4c22a4b5"}
Sep 09 21:52:57.885059 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.885035    2578 scope.go:117] "RemoveContainer" containerID="d16cf1f6c8c15520acc05dfcb622f7cb0f14399c73055886cc190b30ae3448e9"
Sep 09 21:52:57.896282 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.896262    2578 scope.go:117] "RemoveContainer" containerID="c37a9c3589053623cdd5f037f7ffd31a0151e0798bc4a55391254e43aeca38ad"
Sep 09 21:52:57.905668 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.905654    2578 scope.go:117] "RemoveContainer" containerID="d16cf1f6c8c15520acc05dfcb622f7cb0f14399c73055886cc190b30ae3448e9"
Sep 09 21:52:57.905912 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:57.905892    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d16cf1f6c8c15520acc05dfcb622f7cb0f14399c73055886cc190b30ae3448e9\": container with ID starting with d16cf1f6c8c15520acc05dfcb622f7cb0f14399c73055886cc190b30ae3448e9 not found: ID does not exist" containerID="d16cf1f6c8c15520acc05dfcb622f7cb0f14399c73055886cc190b30ae3448e9"
Sep 09 21:52:57.905998 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.905919    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d16cf1f6c8c15520acc05dfcb622f7cb0f14399c73055886cc190b30ae3448e9"} err="failed to get container status \"d16cf1f6c8c15520acc05dfcb622f7cb0f14399c73055886cc190b30ae3448e9\": rpc error: code = NotFound desc = could not find container \"d16cf1f6c8c15520acc05dfcb622f7cb0f14399c73055886cc190b30ae3448e9\": container with ID starting with d16cf1f6c8c15520acc05dfcb622f7cb0f14399c73055886cc190b30ae3448e9 not found: ID does not exist"
Sep 09 21:52:57.905998 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.905934    2578 scope.go:117] "RemoveContainer" containerID="c37a9c3589053623cdd5f037f7ffd31a0151e0798bc4a55391254e43aeca38ad"
Sep 09 21:52:57.906208 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:52:57.906190    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c37a9c3589053623cdd5f037f7ffd31a0151e0798bc4a55391254e43aeca38ad\": container with ID starting with c37a9c3589053623cdd5f037f7ffd31a0151e0798bc4a55391254e43aeca38ad not found: ID does not exist" containerID="c37a9c3589053623cdd5f037f7ffd31a0151e0798bc4a55391254e43aeca38ad"
Sep 09 21:52:57.906276 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.906211    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c37a9c3589053623cdd5f037f7ffd31a0151e0798bc4a55391254e43aeca38ad"} err="failed to get container status \"c37a9c3589053623cdd5f037f7ffd31a0151e0798bc4a55391254e43aeca38ad\": rpc error: code = NotFound desc = could not find container \"c37a9c3589053623cdd5f037f7ffd31a0151e0798bc4a55391254e43aeca38ad\": container with ID starting with c37a9c3589053623cdd5f037f7ffd31a0151e0798bc4a55391254e43aeca38ad not found: ID does not exist"
Sep 09 21:52:57.907863 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.907838    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"]
Sep 09 21:52:57.911945 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:57.911925    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-331c9d-openshift-default-85f6c79d6d-hjzb9"]
Sep 09 21:52:59.005137 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:52:59.005060    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c99ff219-f707-4de1-8299-c7ea5bf1d199" path="/var/lib/kubelet/pods/c99ff219-f707-4de1-8299-c7ea5bf1d199/volumes"
Sep 09 21:53:01.053820 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:01.053790    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 21:53:01.057468 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:01.057440    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 21:53:01.919999 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:01.919916    2578 generic.go:356] "Generic (PLEG): container finished" podID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" containerID="38ef4080a7abc67761a9e9df95a8691596361073041afcf8604483b169c8f5b9" exitCode=0
Sep 09 21:53:01.920167 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:01.919992    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r" event={"ID":"4e09b6ed-f686-4bf8-9aa0-22859f5eed27","Type":"ContainerDied","Data":"38ef4080a7abc67761a9e9df95a8691596361073041afcf8604483b169c8f5b9"}
Sep 09 21:53:01.920415 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:01.920398    2578 scope.go:117] "RemoveContainer" containerID="38ef4080a7abc67761a9e9df95a8691596361073041afcf8604483b169c8f5b9"
Sep 09 21:53:01.921630 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:01.921598    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:53:01.921707 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:01.921668    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:53:01.921707 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:01.921697    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:53:02.928587 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:02.928554    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r" event={"ID":"4e09b6ed-f686-4bf8-9aa0-22859f5eed27","Type":"ContainerStarted","Data":"d66f6b83c1de5a00e482e3b0d524474bbfff3dea5f303df07eff344aefdbc546"}
Sep 09 21:53:03.341221 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:03.341183    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:53:03.341421 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:03.341299    2578 patch_prober.go:28] interesting pod/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 09 21:53:03.341421 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:03.341344    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r" podUID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 09 21:53:04.340861 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:04.340816    2578 patch_prober.go:28] interesting pod/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 09 21:53:04.341342 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:04.340895    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r" podUID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 09 21:53:05.341345 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:05.341301    2578 patch_prober.go:28] interesting pod/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 09 21:53:05.341870 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:05.341382    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r" podUID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 09 21:53:06.341211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:06.341167    2578 patch_prober.go:28] interesting pod/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 09 21:53:06.341404 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:06.341230    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r" podUID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 09 21:53:07.341505 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:07.341450    2578 patch_prober.go:28] interesting pod/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 09 21:53:07.342036 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:07.341550    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r" podUID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 09 21:53:08.345199 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:08.345171    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:53:08.345667 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:08.345647    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:53:08.346388 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:08.346366    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:53:08.871778 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:08.871743    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:53:30.183803 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:30.183770    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:53:38.479546 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:38.479514    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"]
Sep 09 21:53:38.479991 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:38.479740    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth" podUID="adec3dcb-a12f-4d93-83df-dac8b9ebeab2" containerName="maas-api" containerID="cri-o://2e13280c1c2fcd2d746d245b8dcb88abc0255bfc60f8b79f26f01592998540a6" gracePeriod=30
Sep 09 21:53:38.728301 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:38.728278    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:53:38.906600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:38.905284    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-openshift-service-ca\") pod \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") "
Sep 09 21:53:38.906600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:38.905378    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-metrics-certs\") pod \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") "
Sep 09 21:53:38.906600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:38.905402    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-maas-api-tls\") pod \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") "
Sep 09 21:53:38.906600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:38.905539    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mn4nx\" (UniqueName: \"kubernetes.io/projected/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-kube-api-access-mn4nx\") pod \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\" (UID: \"adec3dcb-a12f-4d93-83df-dac8b9ebeab2\") "
Sep 09 21:53:38.906600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:38.906306    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "adec3dcb-a12f-4d93-83df-dac8b9ebeab2" (UID: "adec3dcb-a12f-4d93-83df-dac8b9ebeab2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:53:38.909817 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:38.909721    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-kube-api-access-mn4nx" (OuterVolumeSpecName: "kube-api-access-mn4nx") pod "adec3dcb-a12f-4d93-83df-dac8b9ebeab2" (UID: "adec3dcb-a12f-4d93-83df-dac8b9ebeab2"). InnerVolumeSpecName "kube-api-access-mn4nx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:53:38.910475 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:38.910428    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "adec3dcb-a12f-4d93-83df-dac8b9ebeab2" (UID: "adec3dcb-a12f-4d93-83df-dac8b9ebeab2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:53:38.910714 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:38.910692    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "adec3dcb-a12f-4d93-83df-dac8b9ebeab2" (UID: "adec3dcb-a12f-4d93-83df-dac8b9ebeab2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:53:39.006726 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:39.006696    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn4nx\" (UniqueName: \"kubernetes.io/projected/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-kube-api-access-mn4nx\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:53:39.006726 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:39.006722    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-openshift-service-ca\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:53:39.006726 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:39.006732    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-metrics-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:53:39.006954 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:39.006740    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/adec3dcb-a12f-4d93-83df-dac8b9ebeab2-maas-api-tls\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:53:39.170510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:39.170420    2578 generic.go:356] "Generic (PLEG): container finished" podID="adec3dcb-a12f-4d93-83df-dac8b9ebeab2" containerID="2e13280c1c2fcd2d746d245b8dcb88abc0255bfc60f8b79f26f01592998540a6" exitCode=0
Sep 09 21:53:39.170510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:39.170467    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"
Sep 09 21:53:39.170751 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:39.170515    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth" event={"ID":"adec3dcb-a12f-4d93-83df-dac8b9ebeab2","Type":"ContainerDied","Data":"2e13280c1c2fcd2d746d245b8dcb88abc0255bfc60f8b79f26f01592998540a6"}
Sep 09 21:53:39.170751 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:39.170579    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth" event={"ID":"adec3dcb-a12f-4d93-83df-dac8b9ebeab2","Type":"ContainerDied","Data":"5f14f98f2737532aa4afb4b4df975c6842338a314e2050abce97f28812715e1a"}
Sep 09 21:53:39.170751 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:39.170608    2578 scope.go:117] "RemoveContainer" containerID="2e13280c1c2fcd2d746d245b8dcb88abc0255bfc60f8b79f26f01592998540a6"
Sep 09 21:53:39.180194 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:39.180178    2578 scope.go:117] "RemoveContainer" containerID="2e13280c1c2fcd2d746d245b8dcb88abc0255bfc60f8b79f26f01592998540a6"
Sep 09 21:53:39.180493 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:53:39.180473    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e13280c1c2fcd2d746d245b8dcb88abc0255bfc60f8b79f26f01592998540a6\": container with ID starting with 2e13280c1c2fcd2d746d245b8dcb88abc0255bfc60f8b79f26f01592998540a6 not found: ID does not exist" containerID="2e13280c1c2fcd2d746d245b8dcb88abc0255bfc60f8b79f26f01592998540a6"
Sep 09 21:53:39.180566 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:39.180505    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e13280c1c2fcd2d746d245b8dcb88abc0255bfc60f8b79f26f01592998540a6"} err="failed to get container status \"2e13280c1c2fcd2d746d245b8dcb88abc0255bfc60f8b79f26f01592998540a6\": rpc error: code = NotFound desc = could not find container \"2e13280c1c2fcd2d746d245b8dcb88abc0255bfc60f8b79f26f01592998540a6\": container with ID starting with 2e13280c1c2fcd2d746d245b8dcb88abc0255bfc60f8b79f26f01592998540a6 not found: ID does not exist"
Sep 09 21:53:39.187302 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:39.187256    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"]
Sep 09 21:53:39.190306 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:39.190285    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2aaab1-74b7cf5c56-cdbth"]
Sep 09 21:53:41.004736 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:41.004704    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adec3dcb-a12f-4d93-83df-dac8b9ebeab2" path="/var/lib/kubelet/pods/adec3dcb-a12f-4d93-83df-dac8b9ebeab2/volumes"
Sep 09 21:53:45.084603 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:45.084563    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"]
Sep 09 21:53:45.085124 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:45.084852    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerName="istio-proxy" containerID="cri-o://00619049f7a84b437a89024acb21b0ceda0f98925c8eaa479c2fe743ad8a29a4" gracePeriod=30
Sep 09 21:53:49.202830 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:49.202794    2578 patch_prober.go:28] interesting pod/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 21:53:49.203239 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:49.202858    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:53:50.255543 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.255516    2578 generic.go:356] "Generic (PLEG): container finished" podID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerID="00619049f7a84b437a89024acb21b0ceda0f98925c8eaa479c2fe743ad8a29a4" exitCode=0
Sep 09 21:53:50.255910 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.255591    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" event={"ID":"40c04813-7e10-4669-a048-93ad3f50c5b4","Type":"ContainerDied","Data":"00619049f7a84b437a89024acb21b0ceda0f98925c8eaa479c2fe743ad8a29a4"}
Sep 09 21:53:50.255910 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.255669    2578 scope.go:117] "RemoveContainer" containerID="00b8c89d9eea7e7fe3309f7de78726d9a8cfcc95eb1066167ef353f02f740615"
Sep 09 21:53:50.337657 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.337635    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:53:50.405136 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.405068    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-token\") pod \"40c04813-7e10-4669-a048-93ad3f50c5b4\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") "
Sep 09 21:53:50.405258 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.405136    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-workload-socket\") pod \"40c04813-7e10-4669-a048-93ad3f50c5b4\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") "
Sep 09 21:53:50.405258 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.405204    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-podinfo\") pod \"40c04813-7e10-4669-a048-93ad3f50c5b4\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") "
Sep 09 21:53:50.405258 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.405228    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-credential-socket\") pod \"40c04813-7e10-4669-a048-93ad3f50c5b4\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") "
Sep 09 21:53:50.405429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.405265    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-envoy\") pod \"40c04813-7e10-4669-a048-93ad3f50c5b4\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") "
Sep 09 21:53:50.405429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.405284    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvl6s\" (UniqueName: \"kubernetes.io/projected/40c04813-7e10-4669-a048-93ad3f50c5b4-kube-api-access-fvl6s\") pod \"40c04813-7e10-4669-a048-93ad3f50c5b4\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") "
Sep 09 21:53:50.405429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.405301    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-workload-certs\") pod \"40c04813-7e10-4669-a048-93ad3f50c5b4\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") "
Sep 09 21:53:50.405599 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.405451    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "40c04813-7e10-4669-a048-93ad3f50c5b4" (UID: "40c04813-7e10-4669-a048-93ad3f50c5b4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:53:50.405599 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.405530    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "40c04813-7e10-4669-a048-93ad3f50c5b4" (UID: "40c04813-7e10-4669-a048-93ad3f50c5b4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:53:50.405700 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.405609    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "40c04813-7e10-4669-a048-93ad3f50c5b4" (UID: "40c04813-7e10-4669-a048-93ad3f50c5b4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:53:50.405700 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.405641    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-data\") pod \"40c04813-7e10-4669-a048-93ad3f50c5b4\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") "
Sep 09 21:53:50.405818 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.405694    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c04813-7e10-4669-a048-93ad3f50c5b4-istiod-ca-cert\") pod \"40c04813-7e10-4669-a048-93ad3f50c5b4\" (UID: \"40c04813-7e10-4669-a048-93ad3f50c5b4\") "
Sep 09 21:53:50.405818 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.405757    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-data" (OuterVolumeSpecName: "istio-data") pod "40c04813-7e10-4669-a048-93ad3f50c5b4" (UID: "40c04813-7e10-4669-a048-93ad3f50c5b4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:53:50.406103 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.406077    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40c04813-7e10-4669-a048-93ad3f50c5b4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "40c04813-7e10-4669-a048-93ad3f50c5b4" (UID: "40c04813-7e10-4669-a048-93ad3f50c5b4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:53:50.406103 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.406099    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:53:50.406276 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.406118    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:53:50.406276 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.406132    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:53:50.406276 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.406145    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:53:50.407443 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.407423    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "40c04813-7e10-4669-a048-93ad3f50c5b4" (UID: "40c04813-7e10-4669-a048-93ad3f50c5b4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:53:50.407557 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.407537    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40c04813-7e10-4669-a048-93ad3f50c5b4-kube-api-access-fvl6s" (OuterVolumeSpecName: "kube-api-access-fvl6s") pod "40c04813-7e10-4669-a048-93ad3f50c5b4" (UID: "40c04813-7e10-4669-a048-93ad3f50c5b4"). InnerVolumeSpecName "kube-api-access-fvl6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:53:50.407716 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.407692    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "40c04813-7e10-4669-a048-93ad3f50c5b4" (UID: "40c04813-7e10-4669-a048-93ad3f50c5b4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:53:50.407810 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.407712    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-token" (OuterVolumeSpecName: "istio-token") pod "40c04813-7e10-4669-a048-93ad3f50c5b4" (UID: "40c04813-7e10-4669-a048-93ad3f50c5b4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:53:50.506816 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.506786    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:53:50.506816 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.506812    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:53:50.506816 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.506821    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvl6s\" (UniqueName: \"kubernetes.io/projected/40c04813-7e10-4669-a048-93ad3f50c5b4-kube-api-access-fvl6s\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:53:50.507053 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.506831    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c04813-7e10-4669-a048-93ad3f50c5b4-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:53:50.507053 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:50.506839    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c04813-7e10-4669-a048-93ad3f50c5b4-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:53:51.262523 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:51.262443    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"
Sep 09 21:53:51.262523 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:51.262491    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq" event={"ID":"40c04813-7e10-4669-a048-93ad3f50c5b4","Type":"ContainerDied","Data":"879fcf5b67fb909a3188c333c21ade952b649cb6077ef1b68a4dbcfe21eef3bb"}
Sep 09 21:53:51.262982 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:51.262550    2578 scope.go:117] "RemoveContainer" containerID="00619049f7a84b437a89024acb21b0ceda0f98925c8eaa479c2fe743ad8a29a4"
Sep 09 21:53:51.279733 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:51.279711    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"]
Sep 09 21:53:51.283431 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:51.283407    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-2aaab1-openshift-default-59c66c6874-ztrkq"]
Sep 09 21:53:53.003561 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:53:53.003531    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" path="/var/lib/kubelet/pods/40c04813-7e10-4669-a048-93ad3f50c5b4/volumes"
Sep 09 21:54:04.995419 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:04.995385    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:54:08.993431 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:08.993399    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"]
Sep 09 21:54:08.993872 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:08.993858    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerName="istio-proxy"
Sep 09 21:54:08.993918 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:08.993874    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerName="istio-proxy"
Sep 09 21:54:08.993918 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:08.993885    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c99ff219-f707-4de1-8299-c7ea5bf1d199" containerName="istio-proxy"
Sep 09 21:54:08.993918 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:08.993890    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c99ff219-f707-4de1-8299-c7ea5bf1d199" containerName="istio-proxy"
Sep 09 21:54:08.993918 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:08.993915    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adec3dcb-a12f-4d93-83df-dac8b9ebeab2" containerName="maas-api"
Sep 09 21:54:08.994118 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:08.993921    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="adec3dcb-a12f-4d93-83df-dac8b9ebeab2" containerName="maas-api"
Sep 09 21:54:08.994118 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:08.994019    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerName="istio-proxy"
Sep 09 21:54:08.994118 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:08.994028    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c99ff219-f707-4de1-8299-c7ea5bf1d199" containerName="istio-proxy"
Sep 09 21:54:08.994118 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:08.994036    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="adec3dcb-a12f-4d93-83df-dac8b9ebeab2" containerName="maas-api"
Sep 09 21:54:08.998541 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:08.998519    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.001714 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.001685    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3fa986\"/\"default-dockercfg-h6wml\""
Sep 09 21:54:09.001860 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.001691    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-3fa986\"/\"kube-root-ca.crt\""
Sep 09 21:54:09.001860 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.001694    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3fa986\"/\"distinct-2-w4-3fa986-kserve-self-signed-certs\""
Sep 09 21:54:09.001860 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.001754    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-3fa986\"/\"openshift-service-ca.crt\""
Sep 09 21:54:09.007187 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.007159    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"]
Sep 09 21:54:09.083579 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.083556    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5tm8\" (UniqueName: \"kubernetes.io/projected/48a06290-8726-4a1f-985a-fbfa0923bdb5-kube-api-access-k5tm8\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.083728 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.083597    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-home\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.083728 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.083613    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-tmp-dir\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.083858 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.083721    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-model-cache\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.083858 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.083788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-dshm\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.083962 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.083868    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48a06290-8726-4a1f-985a-fbfa0923bdb5-tls-certs\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.185200 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.185166    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48a06290-8726-4a1f-985a-fbfa0923bdb5-tls-certs\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.185405 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.185238    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5tm8\" (UniqueName: \"kubernetes.io/projected/48a06290-8726-4a1f-985a-fbfa0923bdb5-kube-api-access-k5tm8\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.185405 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.185271    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-home\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.185405 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.185287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-tmp-dir\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.185405 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.185339    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-model-cache\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.185405 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.185385    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-dshm\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.185795 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.185773    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-home\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.185869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.185803    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-tmp-dir\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.185869 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.185827    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-model-cache\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.187575 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.187554    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-dshm\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.187840 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.187824    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48a06290-8726-4a1f-985a-fbfa0923bdb5-tls-certs\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.192466 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.192445    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5tm8\" (UniqueName: \"kubernetes.io/projected/48a06290-8726-4a1f-985a-fbfa0923bdb5-kube-api-access-k5tm8\") pod \"distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") " pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.309738 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.309700    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:09.583253 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:09.583229    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"]
Sep 09 21:54:09.587443 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:54:09.587394    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48a06290_8726_4a1f_985a_fbfa0923bdb5.slice/crio-b157c773039f0d6f0ac4a8971321168bd9c588ebfb1ca1a3a93ddecfc9de9d18 WatchSource:0}: Error finding container b157c773039f0d6f0ac4a8971321168bd9c588ebfb1ca1a3a93ddecfc9de9d18: Status 404 returned error can't find the container with id b157c773039f0d6f0ac4a8971321168bd9c588ebfb1ca1a3a93ddecfc9de9d18
Sep 09 21:54:10.387381 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:10.387347    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh" event={"ID":"48a06290-8726-4a1f-985a-fbfa0923bdb5","Type":"ContainerStarted","Data":"d4dc1bf76c78466aa9fa87e377f0e71dcdf9452bb58b5b8a984609ae9394b4af"}
Sep 09 21:54:10.387381 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:10.387386    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh" event={"ID":"48a06290-8726-4a1f-985a-fbfa0923bdb5","Type":"ContainerStarted","Data":"b157c773039f0d6f0ac4a8971321168bd9c588ebfb1ca1a3a93ddecfc9de9d18"}
Sep 09 21:54:10.405443 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:10.405390    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh" podStartSLOduration=2.4053745810000002 podStartE2EDuration="2.405374581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:54:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:54:10.403842713 +0000 UTC m=+1269.876807622" watchObservedRunningTime="2026-09-09 21:54:10.405374581 +0000 UTC m=+1269.878339491"
Sep 09 21:54:19.310133 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:19.310088    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:19.310530 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:19.310150    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:19.322695 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:19.322660    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:19.452738 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:19.452709    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 21:54:31.779464 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:31.779428    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:54:32.360351 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:32.360319    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"]
Sep 09 21:54:32.360598 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:32.360562    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq" podUID="099567be-4032-48ea-b06a-eed206dc7028" containerName="istio-proxy" containerID="cri-o://60d7aa3e4f850b5b6f73c5f43e1990896b4c437fa839eedc5407a9cde1cef2b1" gracePeriod=30
Sep 09 21:54:37.561310 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.561281    2578 generic.go:356] "Generic (PLEG): container finished" podID="099567be-4032-48ea-b06a-eed206dc7028" containerID="60d7aa3e4f850b5b6f73c5f43e1990896b4c437fa839eedc5407a9cde1cef2b1" exitCode=0
Sep 09 21:54:37.561691 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.561358    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq" event={"ID":"099567be-4032-48ea-b06a-eed206dc7028","Type":"ContainerDied","Data":"60d7aa3e4f850b5b6f73c5f43e1990896b4c437fa839eedc5407a9cde1cef2b1"}
Sep 09 21:54:37.627124 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.627069    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:54:37.660432 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.660397    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/099567be-4032-48ea-b06a-eed206dc7028-istiod-ca-cert\") pod \"099567be-4032-48ea-b06a-eed206dc7028\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") "
Sep 09 21:54:37.660603 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.660444    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/099567be-4032-48ea-b06a-eed206dc7028-istio-podinfo\") pod \"099567be-4032-48ea-b06a-eed206dc7028\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") "
Sep 09 21:54:37.660603 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.660462    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-credential-socket\") pod \"099567be-4032-48ea-b06a-eed206dc7028\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") "
Sep 09 21:54:37.660603 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.660546    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-istio-envoy\") pod \"099567be-4032-48ea-b06a-eed206dc7028\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") "
Sep 09 21:54:37.660603 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.660577    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/099567be-4032-48ea-b06a-eed206dc7028-istio-token\") pod \"099567be-4032-48ea-b06a-eed206dc7028\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") "
Sep 09 21:54:37.660603 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.660602    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-workload-socket\") pod \"099567be-4032-48ea-b06a-eed206dc7028\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") "
Sep 09 21:54:37.660885 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.660653    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-istio-data\") pod \"099567be-4032-48ea-b06a-eed206dc7028\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") "
Sep 09 21:54:37.660885 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.660680    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6fnz\" (UniqueName: \"kubernetes.io/projected/099567be-4032-48ea-b06a-eed206dc7028-kube-api-access-c6fnz\") pod \"099567be-4032-48ea-b06a-eed206dc7028\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") "
Sep 09 21:54:37.660885 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.660706    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-workload-certs\") pod \"099567be-4032-48ea-b06a-eed206dc7028\" (UID: \"099567be-4032-48ea-b06a-eed206dc7028\") "
Sep 09 21:54:37.660885 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.660847    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "099567be-4032-48ea-b06a-eed206dc7028" (UID: "099567be-4032-48ea-b06a-eed206dc7028"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:54:37.660885 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.660855    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/099567be-4032-48ea-b06a-eed206dc7028-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "099567be-4032-48ea-b06a-eed206dc7028" (UID: "099567be-4032-48ea-b06a-eed206dc7028"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:54:37.661211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.661029    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "099567be-4032-48ea-b06a-eed206dc7028" (UID: "099567be-4032-48ea-b06a-eed206dc7028"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:54:37.661274 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.661210    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:37.661274 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.661231    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/099567be-4032-48ea-b06a-eed206dc7028-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:37.661274 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.661244    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:37.661274 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.661261    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "099567be-4032-48ea-b06a-eed206dc7028" (UID: "099567be-4032-48ea-b06a-eed206dc7028"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:54:37.661489 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.661449    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-istio-data" (OuterVolumeSpecName: "istio-data") pod "099567be-4032-48ea-b06a-eed206dc7028" (UID: "099567be-4032-48ea-b06a-eed206dc7028"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:54:37.663198 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.663172    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/099567be-4032-48ea-b06a-eed206dc7028-kube-api-access-c6fnz" (OuterVolumeSpecName: "kube-api-access-c6fnz") pod "099567be-4032-48ea-b06a-eed206dc7028" (UID: "099567be-4032-48ea-b06a-eed206dc7028"). InnerVolumeSpecName "kube-api-access-c6fnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:54:37.663315 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.663275    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/099567be-4032-48ea-b06a-eed206dc7028-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "099567be-4032-48ea-b06a-eed206dc7028" (UID: "099567be-4032-48ea-b06a-eed206dc7028"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:54:37.663366 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.663336    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/099567be-4032-48ea-b06a-eed206dc7028-istio-token" (OuterVolumeSpecName: "istio-token") pod "099567be-4032-48ea-b06a-eed206dc7028" (UID: "099567be-4032-48ea-b06a-eed206dc7028"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:54:37.663589 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.663567    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "099567be-4032-48ea-b06a-eed206dc7028" (UID: "099567be-4032-48ea-b06a-eed206dc7028"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:54:37.762122 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.762092    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/099567be-4032-48ea-b06a-eed206dc7028-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:37.762122 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.762119    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:37.762122 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.762128    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/099567be-4032-48ea-b06a-eed206dc7028-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:37.762350 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.762135    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:37.762350 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.762143    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6fnz\" (UniqueName: \"kubernetes.io/projected/099567be-4032-48ea-b06a-eed206dc7028-kube-api-access-c6fnz\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:37.762350 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:37.762152    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/099567be-4032-48ea-b06a-eed206dc7028-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:38.354538 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:38.354502    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"]
Sep 09 21:54:38.354766 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:38.354718    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5" podUID="e5ca4200-2da3-459c-9feb-deeb311d1d70" containerName="istio-proxy" containerID="cri-o://2471b4508d082f5b2aeda930c44f5bade703f32fe8794cb68f1a7071f7cea217" gracePeriod=30
Sep 09 21:54:38.568864 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:38.568831    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq" event={"ID":"099567be-4032-48ea-b06a-eed206dc7028","Type":"ContainerDied","Data":"92c165c8e0df01630faf2b56319b4dda8781f6217c25d7093dfe42d0046eea74"}
Sep 09 21:54:38.568864 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:38.568864    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"
Sep 09 21:54:38.569326 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:38.568890    2578 scope.go:117] "RemoveContainer" containerID="60d7aa3e4f850b5b6f73c5f43e1990896b4c437fa839eedc5407a9cde1cef2b1"
Sep 09 21:54:38.593624 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:38.593599    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"]
Sep 09 21:54:38.597267 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:38.597243    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-749a53-openshift-default-7c4cbc8bb6-h2hgq"]
Sep 09 21:54:39.004895 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.004865    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="099567be-4032-48ea-b06a-eed206dc7028" path="/var/lib/kubelet/pods/099567be-4032-48ea-b06a-eed206dc7028/volumes"
Sep 09 21:54:39.161726 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.161695    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"]
Sep 09 21:54:39.162376 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.162356    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099567be-4032-48ea-b06a-eed206dc7028" containerName="istio-proxy"
Sep 09 21:54:39.162422 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.162381    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="099567be-4032-48ea-b06a-eed206dc7028" containerName="istio-proxy"
Sep 09 21:54:39.162467 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.162437    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerName="istio-proxy"
Sep 09 21:54:39.162467 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.162446    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerName="istio-proxy"
Sep 09 21:54:39.162467 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.162459    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c99ff219-f707-4de1-8299-c7ea5bf1d199" containerName="istio-proxy"
Sep 09 21:54:39.162582 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.162467    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c99ff219-f707-4de1-8299-c7ea5bf1d199" containerName="istio-proxy"
Sep 09 21:54:39.162582 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.162557    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="40c04813-7e10-4669-a048-93ad3f50c5b4" containerName="istio-proxy"
Sep 09 21:54:39.162582 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.162567    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c99ff219-f707-4de1-8299-c7ea5bf1d199" containerName="istio-proxy"
Sep 09 21:54:39.162582 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.162578    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="099567be-4032-48ea-b06a-eed206dc7028" containerName="istio-proxy"
Sep 09 21:54:39.167199 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.167181    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.169639 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.169619    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-2d3bc8-openshift-default-dockercfg-5fjc2\""
Sep 09 21:54:39.176455 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.175862    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"]
Sep 09 21:54:39.283167 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.282543    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-credential-socket\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.283167 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.282597    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-token\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.283167 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.282685    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vl8v\" (UniqueName: \"kubernetes.io/projected/6d131295-a2b3-4629-a34e-172b8e8bb828-kube-api-access-9vl8v\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.283167 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.282759    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-workload-certs\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.283167 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.282788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-data\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.283167 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.282820    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-workload-socket\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.283167 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.282847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d131295-a2b3-4629-a34e-172b8e8bb828-istiod-ca-cert\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.283167 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.282893    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-envoy\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.283167 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.282938    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-podinfo\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.384491 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.384454    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-envoy\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.384668 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.384509    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-podinfo\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.384668 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.384565    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-credential-socket\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.384668 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.384581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-token\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.384668 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.384633    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vl8v\" (UniqueName: \"kubernetes.io/projected/6d131295-a2b3-4629-a34e-172b8e8bb828-kube-api-access-9vl8v\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.384668 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.384669    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-workload-certs\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.385065 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.384692    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-data\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.385065 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.384722    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-workload-socket\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.385065 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.384780    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d131295-a2b3-4629-a34e-172b8e8bb828-istiod-ca-cert\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.385065 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.384984    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-credential-socket\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.385289 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.385181    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-data\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.385289 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.385275    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-workload-certs\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.385391 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.385370    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-workload-socket\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.385540 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.385522    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d131295-a2b3-4629-a34e-172b8e8bb828-istiod-ca-cert\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.386840 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.386818    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-envoy\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.387068 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.387051    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-podinfo\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.392230 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.392209    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-token\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.392434 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.392415    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vl8v\" (UniqueName: \"kubernetes.io/projected/6d131295-a2b3-4629-a34e-172b8e8bb828-kube-api-access-9vl8v\") pod \"e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") " pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.479927 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.479896    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:39.759514 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.759482    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"]
Sep 09 21:54:39.760701 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:54:39.760676    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d131295_a2b3_4629_a34e_172b8e8bb828.slice/crio-41401836dc3f26d48bd72e35c5b6299a2edea733247911a83932da55af9e5a65 WatchSource:0}: Error finding container 41401836dc3f26d48bd72e35c5b6299a2edea733247911a83932da55af9e5a65: Status 404 returned error can't find the container with id 41401836dc3f26d48bd72e35c5b6299a2edea733247911a83932da55af9e5a65
Sep 09 21:54:39.763466 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.763432    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:54:39.763540 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.763498    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:54:39.763540 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:39.763527    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:54:40.586104 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:40.586070    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r" event={"ID":"6d131295-a2b3-4629-a34e-172b8e8bb828","Type":"ContainerStarted","Data":"204c082d1ac5a269f97831cd1d00162077b7e08b48045d566b7ccdea6005f6d3"}
Sep 09 21:54:40.586363 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:40.586123    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r" event={"ID":"6d131295-a2b3-4629-a34e-172b8e8bb828","Type":"ContainerStarted","Data":"41401836dc3f26d48bd72e35c5b6299a2edea733247911a83932da55af9e5a65"}
Sep 09 21:54:40.607164 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:40.607122    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r" podStartSLOduration=1.607107093 podStartE2EDuration="1.607107093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:54:40.603605041 +0000 UTC m=+1300.076569953" watchObservedRunningTime="2026-09-09 21:54:40.607107093 +0000 UTC m=+1300.080072003"
Sep 09 21:54:41.480363 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:41.480325    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:41.485678 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:41.485654    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:41.592769 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:41.592742    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:41.593596 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:41.593577    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:54:43.606234 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.606202    2578 generic.go:356] "Generic (PLEG): container finished" podID="e5ca4200-2da3-459c-9feb-deeb311d1d70" containerID="2471b4508d082f5b2aeda930c44f5bade703f32fe8794cb68f1a7071f7cea217" exitCode=0
Sep 09 21:54:43.606653 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.606268    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5" event={"ID":"e5ca4200-2da3-459c-9feb-deeb311d1d70","Type":"ContainerDied","Data":"2471b4508d082f5b2aeda930c44f5bade703f32fe8794cb68f1a7071f7cea217"}
Sep 09 21:54:43.606653 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.606324    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5" event={"ID":"e5ca4200-2da3-459c-9feb-deeb311d1d70","Type":"ContainerDied","Data":"066219fd261c5d51ae17e949e534059d79b49ebf34f8e6963bd7f4cb796dbdcc"}
Sep 09 21:54:43.606653 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.606342    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="066219fd261c5d51ae17e949e534059d79b49ebf34f8e6963bd7f4cb796dbdcc"
Sep 09 21:54:43.606653 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.606355    2578 scope.go:117] "RemoveContainer" containerID="07dc140b9ab31425fd4232aab3a6e1ba469681ecbf6bd0b38d2736c48e95eb9f"
Sep 09 21:54:43.619686 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.619641    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:54:43.679582 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.679548    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"]
Sep 09 21:54:43.680058 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.680039    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ca4200-2da3-459c-9feb-deeb311d1d70" containerName="istio-proxy"
Sep 09 21:54:43.680058 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.680056    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ca4200-2da3-459c-9feb-deeb311d1d70" containerName="istio-proxy"
Sep 09 21:54:43.680204 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.680143    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5ca4200-2da3-459c-9feb-deeb311d1d70" containerName="istio-proxy"
Sep 09 21:54:43.680204 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.680152    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5ca4200-2da3-459c-9feb-deeb311d1d70" containerName="istio-proxy"
Sep 09 21:54:43.683567 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.683547    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:43.698292 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.698272    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-2d3bc8-serving-cert\""
Sep 09 21:54:43.711646 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.711620    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"]
Sep 09 21:54:43.729322 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.729295    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-credential-socket\") pod \"e5ca4200-2da3-459c-9feb-deeb311d1d70\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") "
Sep 09 21:54:43.729457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.729358    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-workload-certs\") pod \"e5ca4200-2da3-459c-9feb-deeb311d1d70\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") "
Sep 09 21:54:43.729457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.729389    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-token\") pod \"e5ca4200-2da3-459c-9feb-deeb311d1d70\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") "
Sep 09 21:54:43.729457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.729427    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-podinfo\") pod \"e5ca4200-2da3-459c-9feb-deeb311d1d70\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") "
Sep 09 21:54:43.729457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.729453    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5ca4200-2da3-459c-9feb-deeb311d1d70-istiod-ca-cert\") pod \"e5ca4200-2da3-459c-9feb-deeb311d1d70\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") "
Sep 09 21:54:43.729700 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.729477    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-envoy\") pod \"e5ca4200-2da3-459c-9feb-deeb311d1d70\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") "
Sep 09 21:54:43.729700 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.729520    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-data\") pod \"e5ca4200-2da3-459c-9feb-deeb311d1d70\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") "
Sep 09 21:54:43.729700 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.729574    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-workload-socket\") pod \"e5ca4200-2da3-459c-9feb-deeb311d1d70\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") "
Sep 09 21:54:43.729700 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.729688    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqc9s\" (UniqueName: \"kubernetes.io/projected/e5ca4200-2da3-459c-9feb-deeb311d1d70-kube-api-access-fqc9s\") pod \"e5ca4200-2da3-459c-9feb-deeb311d1d70\" (UID: \"e5ca4200-2da3-459c-9feb-deeb311d1d70\") "
Sep 09 21:54:43.730358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.729737    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e5ca4200-2da3-459c-9feb-deeb311d1d70" (UID: "e5ca4200-2da3-459c-9feb-deeb311d1d70"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:54:43.730358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.729760    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e5ca4200-2da3-459c-9feb-deeb311d1d70" (UID: "e5ca4200-2da3-459c-9feb-deeb311d1d70"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:54:43.730358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.729886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce598ded-25f8-44d8-8285-e8c0122f1e21-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:43.730358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.729941    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-data" (OuterVolumeSpecName: "istio-data") pod "e5ca4200-2da3-459c-9feb-deeb311d1d70" (UID: "e5ca4200-2da3-459c-9feb-deeb311d1d70"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:54:43.730358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.729954    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnjl5\" (UniqueName: \"kubernetes.io/projected/ce598ded-25f8-44d8-8285-e8c0122f1e21-kube-api-access-lnjl5\") pod \"maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:43.730358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.729962    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e5ca4200-2da3-459c-9feb-deeb311d1d70" (UID: "e5ca4200-2da3-459c-9feb-deeb311d1d70"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:54:43.730358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.730013    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce598ded-25f8-44d8-8285-e8c0122f1e21-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:43.730358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.730249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce598ded-25f8-44d8-8285-e8c0122f1e21-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:43.730358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.730326    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:43.730358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.730342    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:43.730358 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.730357    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:43.732300 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.730369    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:43.732300 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.730404    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5ca4200-2da3-459c-9feb-deeb311d1d70-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e5ca4200-2da3-459c-9feb-deeb311d1d70" (UID: "e5ca4200-2da3-459c-9feb-deeb311d1d70"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:54:43.732409 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.732391    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e5ca4200-2da3-459c-9feb-deeb311d1d70" (UID: "e5ca4200-2da3-459c-9feb-deeb311d1d70"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:54:43.732941 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.732904    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-token" (OuterVolumeSpecName: "istio-token") pod "e5ca4200-2da3-459c-9feb-deeb311d1d70" (UID: "e5ca4200-2da3-459c-9feb-deeb311d1d70"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:54:43.733115 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.733090    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5ca4200-2da3-459c-9feb-deeb311d1d70-kube-api-access-fqc9s" (OuterVolumeSpecName: "kube-api-access-fqc9s") pod "e5ca4200-2da3-459c-9feb-deeb311d1d70" (UID: "e5ca4200-2da3-459c-9feb-deeb311d1d70"). InnerVolumeSpecName "kube-api-access-fqc9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:54:43.733689 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.733656    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e5ca4200-2da3-459c-9feb-deeb311d1d70" (UID: "e5ca4200-2da3-459c-9feb-deeb311d1d70"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:54:43.831380 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.831341    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce598ded-25f8-44d8-8285-e8c0122f1e21-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:43.831585 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.831408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce598ded-25f8-44d8-8285-e8c0122f1e21-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:43.831585 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.831456    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnjl5\" (UniqueName: \"kubernetes.io/projected/ce598ded-25f8-44d8-8285-e8c0122f1e21-kube-api-access-lnjl5\") pod \"maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:43.831585 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.831487    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce598ded-25f8-44d8-8285-e8c0122f1e21-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:43.831760 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.831592    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqc9s\" (UniqueName: \"kubernetes.io/projected/e5ca4200-2da3-459c-9feb-deeb311d1d70-kube-api-access-fqc9s\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:43.831760 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.831608    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:43.831760 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.831622    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:43.831760 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.831635    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5ca4200-2da3-459c-9feb-deeb311d1d70-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:43.831760 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.831649    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5ca4200-2da3-459c-9feb-deeb311d1d70-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:54:43.831760 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:54:43.831747    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-serving-cert: secret "maas-api-e2e-shared-a-w3-2d3bc8-serving-cert" not found
Sep 09 21:54:43.832069 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:54:43.831827    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce598ded-25f8-44d8-8285-e8c0122f1e21-maas-api-tls podName:ce598ded-25f8-44d8-8285-e8c0122f1e21 nodeName:}" failed. No retries permitted until 2026-09-09 21:54:44.331804637 +0000 UTC m=+1303.804769542 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ce598ded-25f8-44d8-8285-e8c0122f1e21-maas-api-tls") pod "maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289" (UID: "ce598ded-25f8-44d8-8285-e8c0122f1e21") : secret "maas-api-e2e-shared-a-w3-2d3bc8-serving-cert" not found
Sep 09 21:54:43.832174 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.832150    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce598ded-25f8-44d8-8285-e8c0122f1e21-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:43.834012 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.833962    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce598ded-25f8-44d8-8285-e8c0122f1e21-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:43.840819 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:43.840790    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnjl5\" (UniqueName: \"kubernetes.io/projected/ce598ded-25f8-44d8-8285-e8c0122f1e21-kube-api-access-lnjl5\") pod \"maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:44.336827 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:44.336795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce598ded-25f8-44d8-8285-e8c0122f1e21-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:44.339530 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:44.339498    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce598ded-25f8-44d8-8285-e8c0122f1e21-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:44.596251 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:44.596101    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:44.619841 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:44.619760    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"
Sep 09 21:54:44.661666 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:44.661585    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"]
Sep 09 21:54:44.665804 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:44.665780    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-da0a18-openshift-default-75799b96b-bjpl5"]
Sep 09 21:54:44.898874 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:44.898847    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"]
Sep 09 21:54:44.900152 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:54:44.900125    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce598ded_25f8_44d8_8285_e8c0122f1e21.slice/crio-05a102e40d28512e931e76467da492dde1bdd6fead9f4773126a0ebd9dc46816 WatchSource:0}: Error finding container 05a102e40d28512e931e76467da492dde1bdd6fead9f4773126a0ebd9dc46816: Status 404 returned error can't find the container with id 05a102e40d28512e931e76467da492dde1bdd6fead9f4773126a0ebd9dc46816
Sep 09 21:54:45.004746 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:45.004717    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5ca4200-2da3-459c-9feb-deeb311d1d70" path="/var/lib/kubelet/pods/e5ca4200-2da3-459c-9feb-deeb311d1d70/volumes"
Sep 09 21:54:45.629211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:45.629168    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289" event={"ID":"ce598ded-25f8-44d8-8285-e8c0122f1e21","Type":"ContainerStarted","Data":"29acde326e180152d6ff92a012f1592551a9c91240487c9e5129c7c106e3a179"}
Sep 09 21:54:45.629611 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:45.629235    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289" event={"ID":"ce598ded-25f8-44d8-8285-e8c0122f1e21","Type":"ContainerStarted","Data":"05a102e40d28512e931e76467da492dde1bdd6fead9f4773126a0ebd9dc46816"}
Sep 09 21:54:45.629611 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:45.629310    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:45.648287 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:45.648208    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289" podStartSLOduration=2.16296673 podStartE2EDuration="2.648193719s" totalImagesPullingTime="485.226989ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:54:45.646006544 +0000 UTC m=+1305.118971453" watchObservedRunningTime="2026-09-09 21:54:45.648193719 +0000 UTC m=+1305.121158629"
Sep 09 21:54:51.641399 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:51.641369    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 21:54:55.272127 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:54:55.272093    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:55:01.221117 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:01.221087    2578 scope.go:117] "RemoveContainer" containerID="2471b4508d082f5b2aeda930c44f5bade703f32fe8794cb68f1a7071f7cea217"
Sep 09 21:55:01.230708 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:01.230686    2578 scope.go:117] "RemoveContainer" containerID="bbfbfbd938e08e3e530271e85f78ea4c9dbf682c2be0ec2e091dbabe606c82bf"
Sep 09 21:55:05.754376 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:05.754279    2578 generic.go:356] "Generic (PLEG): container finished" podID="6d131295-a2b3-4629-a34e-172b8e8bb828" containerID="204c082d1ac5a269f97831cd1d00162077b7e08b48045d566b7ccdea6005f6d3" exitCode=0
Sep 09 21:55:05.754376 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:05.754336    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r" event={"ID":"6d131295-a2b3-4629-a34e-172b8e8bb828","Type":"ContainerDied","Data":"204c082d1ac5a269f97831cd1d00162077b7e08b48045d566b7ccdea6005f6d3"}
Sep 09 21:55:05.754864 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:05.754727    2578 scope.go:117] "RemoveContainer" containerID="204c082d1ac5a269f97831cd1d00162077b7e08b48045d566b7ccdea6005f6d3"
Sep 09 21:55:05.755857 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:05.755797    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:55:05.755919 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:05.755906    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:55:05.755956 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:05.755944    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:55:06.766839 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:06.766806    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r" event={"ID":"6d131295-a2b3-4629-a34e-172b8e8bb828","Type":"ContainerStarted","Data":"7b6c5aff354741df7bec578f53c97e138826adae81884b4eb8d16e15a4255bca"}
Sep 09 21:55:07.481058 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:07.481019    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:55:07.482453 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:07.482429    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 09 21:55:07.482530 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:07.482483    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r" podUID="6d131295-a2b3-4629-a34e-172b8e8bb828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 09 21:55:07.964672 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:07.964634    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"]
Sep 09 21:55:07.965377 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:07.965355    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ca4200-2da3-459c-9feb-deeb311d1d70" containerName="istio-proxy"
Sep 09 21:55:07.965377 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:07.965378    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ca4200-2da3-459c-9feb-deeb311d1d70" containerName="istio-proxy"
Sep 09 21:55:07.968791 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:07.968767    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:07.971168 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:07.971145    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-64fec6-openshift-default-dockercfg-pg5lx\""
Sep 09 21:55:07.976798 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:07.976776    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"]
Sep 09 21:55:08.080994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.080715    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-token\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.080994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.080847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-workload-certs\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.080994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.080878    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-workload-socket\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.080994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.080912    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-data\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.080994 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.080952    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn9jw\" (UniqueName: \"kubernetes.io/projected/c5ac6049-fc4d-420d-b1f4-cb5788d49023-kube-api-access-bn9jw\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.081541 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.081201    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-envoy\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.081541 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.081245    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-podinfo\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.081541 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.081282    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-credential-socket\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.081541 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.081309    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istiod-ca-cert\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.181913 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.181881    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-envoy\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.182084 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.181921    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-podinfo\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.182084 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.182031    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-credential-socket\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.182177 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.182080    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istiod-ca-cert\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.182177 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.182143    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-token\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.182284 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.182220    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-workload-certs\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.182284 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.182250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-workload-socket\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.182393 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.182285    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-data\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.182393 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.182327    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn9jw\" (UniqueName: \"kubernetes.io/projected/c5ac6049-fc4d-420d-b1f4-cb5788d49023-kube-api-access-bn9jw\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.182548 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.182517    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-credential-socket\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.182690 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.182667    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-workload-socket\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.182836 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.182813    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-workload-certs\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.182911 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.182844    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-data\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.182911 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.182866    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istiod-ca-cert\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.184290 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.184266    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-envoy\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.184768 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.184748    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-podinfo\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.190315 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.190296    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-token\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.190590 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.190572    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn9jw\" (UniqueName: \"kubernetes.io/projected/c5ac6049-fc4d-420d-b1f4-cb5788d49023-kube-api-access-bn9jw\") pod \"e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") " pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.281429 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.281390    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:08.480840 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.480799    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 09 21:55:08.481068 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.480862    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r" podUID="6d131295-a2b3-4629-a34e-172b8e8bb828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 09 21:55:08.582566 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.582541    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"]
Sep 09 21:55:08.584522 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:55:08.584481    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5ac6049_fc4d_420d_b1f4_cb5788d49023.slice/crio-a72c215ad3885efb2a1e54a0de710cabe62e24547eb949a5187ff7b4b6db0ea0 WatchSource:0}: Error finding container a72c215ad3885efb2a1e54a0de710cabe62e24547eb949a5187ff7b4b6db0ea0: Status 404 returned error can't find the container with id a72c215ad3885efb2a1e54a0de710cabe62e24547eb949a5187ff7b4b6db0ea0
Sep 09 21:55:08.587021 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.586979    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:55:08.587114 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.587074    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:55:08.587154 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.587120    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:55:08.783582 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.783503    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2" event={"ID":"c5ac6049-fc4d-420d-b1f4-cb5788d49023","Type":"ContainerStarted","Data":"c864142a0916eed295f79e8bb6a155f3dd65d95f4290da4490b2ac39668755fb"}
Sep 09 21:55:08.783582 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.783571    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2" event={"ID":"c5ac6049-fc4d-420d-b1f4-cb5788d49023","Type":"ContainerStarted","Data":"a72c215ad3885efb2a1e54a0de710cabe62e24547eb949a5187ff7b4b6db0ea0"}
Sep 09 21:55:08.807321 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:08.807272    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2" podStartSLOduration=1.8072573790000002 podStartE2EDuration="1.807257379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:55:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:55:08.803387657 +0000 UTC m=+1328.276352566" watchObservedRunningTime="2026-09-09 21:55:08.807257379 +0000 UTC m=+1328.280222288"
Sep 09 21:55:09.281604 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:09.281568    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:09.286839 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:09.286814    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:09.480486 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:09.480451    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 09 21:55:09.480679 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:09.480511    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r" podUID="6d131295-a2b3-4629-a34e-172b8e8bb828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 09 21:55:09.789672 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:09.789638    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:09.790593 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:09.790573    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:10.480397 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:10.480359    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 09 21:55:10.480785 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:10.480418    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r" podUID="6d131295-a2b3-4629-a34e-172b8e8bb828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 09 21:55:11.481037 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:11.481001    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Sep 09 21:55:11.481466 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:11.481060    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r" podUID="6d131295-a2b3-4629-a34e-172b8e8bb828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Sep 09 21:55:11.593364 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:11.593332    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:55:12.485207 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:12.485176    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:55:12.486236 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:12.486214    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 21:55:25.078002 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:25.077892    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:55:27.217469 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.217439    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"]
Sep 09 21:55:27.217893 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.217646    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j" podUID="d9a6d61e-022e-4117-8299-53981a03c554" containerName="maas-api" containerID="cri-o://021080e57bc15a1d60748a210c6b77ddbc910fb79cb031784eb8c76f605b7bcb" gracePeriod=30
Sep 09 21:55:27.463032 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.463010    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:55:27.579556 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.579524    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlvmz\" (UniqueName: \"kubernetes.io/projected/d9a6d61e-022e-4117-8299-53981a03c554-kube-api-access-tlvmz\") pod \"d9a6d61e-022e-4117-8299-53981a03c554\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") "
Sep 09 21:55:27.579556 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.579555    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a6d61e-022e-4117-8299-53981a03c554-openshift-service-ca\") pod \"d9a6d61e-022e-4117-8299-53981a03c554\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") "
Sep 09 21:55:27.579808 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.579593    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9a6d61e-022e-4117-8299-53981a03c554-maas-api-tls\") pod \"d9a6d61e-022e-4117-8299-53981a03c554\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") "
Sep 09 21:55:27.579808 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.579632    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9a6d61e-022e-4117-8299-53981a03c554-metrics-certs\") pod \"d9a6d61e-022e-4117-8299-53981a03c554\" (UID: \"d9a6d61e-022e-4117-8299-53981a03c554\") "
Sep 09 21:55:27.579990 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.579953    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9a6d61e-022e-4117-8299-53981a03c554-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9a6d61e-022e-4117-8299-53981a03c554" (UID: "d9a6d61e-022e-4117-8299-53981a03c554"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:55:27.580161 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.580137    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a6d61e-022e-4117-8299-53981a03c554-openshift-service-ca\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:55:27.581743 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.581717    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a6d61e-022e-4117-8299-53981a03c554-kube-api-access-tlvmz" (OuterVolumeSpecName: "kube-api-access-tlvmz") pod "d9a6d61e-022e-4117-8299-53981a03c554" (UID: "d9a6d61e-022e-4117-8299-53981a03c554"). InnerVolumeSpecName "kube-api-access-tlvmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:55:27.582225 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.582198    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9a6d61e-022e-4117-8299-53981a03c554-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9a6d61e-022e-4117-8299-53981a03c554" (UID: "d9a6d61e-022e-4117-8299-53981a03c554"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:55:27.582298 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.582242    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9a6d61e-022e-4117-8299-53981a03c554-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d9a6d61e-022e-4117-8299-53981a03c554" (UID: "d9a6d61e-022e-4117-8299-53981a03c554"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:55:27.681014 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.680949    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlvmz\" (UniqueName: \"kubernetes.io/projected/d9a6d61e-022e-4117-8299-53981a03c554-kube-api-access-tlvmz\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:55:27.681014 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.681011    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9a6d61e-022e-4117-8299-53981a03c554-maas-api-tls\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:55:27.681014 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.681021    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9a6d61e-022e-4117-8299-53981a03c554-metrics-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:55:27.906058 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.905942    2578 generic.go:356] "Generic (PLEG): container finished" podID="d9a6d61e-022e-4117-8299-53981a03c554" containerID="021080e57bc15a1d60748a210c6b77ddbc910fb79cb031784eb8c76f605b7bcb" exitCode=0
Sep 09 21:55:27.906058 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.906016    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"
Sep 09 21:55:27.906058 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.906033    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j" event={"ID":"d9a6d61e-022e-4117-8299-53981a03c554","Type":"ContainerDied","Data":"021080e57bc15a1d60748a210c6b77ddbc910fb79cb031784eb8c76f605b7bcb"}
Sep 09 21:55:27.906332 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.906092    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j" event={"ID":"d9a6d61e-022e-4117-8299-53981a03c554","Type":"ContainerDied","Data":"06b25023d35566ef28731cd54ec2d1daaa3b90037521b47918d403eadeaa5d53"}
Sep 09 21:55:27.906332 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.906120    2578 scope.go:117] "RemoveContainer" containerID="021080e57bc15a1d60748a210c6b77ddbc910fb79cb031784eb8c76f605b7bcb"
Sep 09 21:55:27.916151 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.916135    2578 scope.go:117] "RemoveContainer" containerID="021080e57bc15a1d60748a210c6b77ddbc910fb79cb031784eb8c76f605b7bcb"
Sep 09 21:55:27.916417 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:55:27.916400    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"021080e57bc15a1d60748a210c6b77ddbc910fb79cb031784eb8c76f605b7bcb\": container with ID starting with 021080e57bc15a1d60748a210c6b77ddbc910fb79cb031784eb8c76f605b7bcb not found: ID does not exist" containerID="021080e57bc15a1d60748a210c6b77ddbc910fb79cb031784eb8c76f605b7bcb"
Sep 09 21:55:27.916493 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.916425    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"021080e57bc15a1d60748a210c6b77ddbc910fb79cb031784eb8c76f605b7bcb"} err="failed to get container status \"021080e57bc15a1d60748a210c6b77ddbc910fb79cb031784eb8c76f605b7bcb\": rpc error: code = NotFound desc = could not find container \"021080e57bc15a1d60748a210c6b77ddbc910fb79cb031784eb8c76f605b7bcb\": container with ID starting with 021080e57bc15a1d60748a210c6b77ddbc910fb79cb031784eb8c76f605b7bcb not found: ID does not exist"
Sep 09 21:55:27.937289 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.937253    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"]
Sep 09 21:55:27.947634 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:27.947589    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4883373b-58688cdc7b-kwc6j"]
Sep 09 21:55:29.003258 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:29.003228    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9a6d61e-022e-4117-8299-53981a03c554" path="/var/lib/kubelet/pods/d9a6d61e-022e-4117-8299-53981a03c554/volumes"
Sep 09 21:55:36.980313 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:36.980279    2578 generic.go:356] "Generic (PLEG): container finished" podID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" containerID="c864142a0916eed295f79e8bb6a155f3dd65d95f4290da4490b2ac39668755fb" exitCode=0
Sep 09 21:55:36.980811 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:36.980350    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2" event={"ID":"c5ac6049-fc4d-420d-b1f4-cb5788d49023","Type":"ContainerDied","Data":"c864142a0916eed295f79e8bb6a155f3dd65d95f4290da4490b2ac39668755fb"}
Sep 09 21:55:36.980811 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:36.980717    2578 scope.go:117] "RemoveContainer" containerID="c864142a0916eed295f79e8bb6a155f3dd65d95f4290da4490b2ac39668755fb"
Sep 09 21:55:36.981920 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:36.981891    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:55:36.982013 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:36.981962    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:55:36.982059 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:36.982014    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:55:37.987853 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:37.987813    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2" event={"ID":"c5ac6049-fc4d-420d-b1f4-cb5788d49023","Type":"ContainerStarted","Data":"6e16bdcb5f057677bb55ead46c65a3b0197746917be1e79fac3a6f956bdc783f"}
Sep 09 21:55:38.281530 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:38.281495    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:38.282836 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:38.282786    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 09 21:55:38.283007 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:38.282854    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2" podUID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 09 21:55:39.282029 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:39.281995    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 09 21:55:39.282522 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:39.282056    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2" podUID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 09 21:55:39.789850 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:39.789799    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:40.282352 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:40.282320    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 09 21:55:40.282732 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:40.282376    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2" podUID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 09 21:55:41.282462 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:41.282426    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 09 21:55:41.282943 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:41.282500    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2" podUID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 09 21:55:42.282614 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:42.282578    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 09 21:55:42.283139 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:42.282652    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2" podUID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 09 21:55:43.286608 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:43.286576    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:43.287714 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:43.287694    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 21:55:49.274607 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:55:49.274570    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:56:14.994616 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:14.994572    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:56:48.084952 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:48.084909    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:56:51.421019 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:51.420989    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"]
Sep 09 21:56:51.421448 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:51.421197    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r" podUID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" containerName="istio-proxy" containerID="cri-o://d66f6b83c1de5a00e482e3b0d524474bbfff3dea5f303df07eff344aefdbc546" gracePeriod=30
Sep 09 21:56:53.348122 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:53.348084    2578 patch_prober.go:28] interesting pod/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 21:56:53.348503 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:53.348143    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r" podUID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:56:56.690039 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.689953    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:56:56.774444 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.774404    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istiod-ca-cert\") pod \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") "
Sep 09 21:56:56.774609 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.774452    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-data\") pod \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") "
Sep 09 21:56:56.774609 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.774510    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-workload-socket\") pod \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") "
Sep 09 21:56:56.774609 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.774562    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-envoy\") pod \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") "
Sep 09 21:56:56.774737 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.774625    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wskfn\" (UniqueName: \"kubernetes.io/projected/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-kube-api-access-wskfn\") pod \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") "
Sep 09 21:56:56.774737 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.774675    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-workload-certs\") pod \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") "
Sep 09 21:56:56.774737 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.774727    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-token\") pod \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") "
Sep 09 21:56:56.774892 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.774752    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-podinfo\") pod \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") "
Sep 09 21:56:56.774892 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.774777    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-credential-socket\") pod \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\" (UID: \"4e09b6ed-f686-4bf8-9aa0-22859f5eed27\") "
Sep 09 21:56:56.774892 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.774786    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-data" (OuterVolumeSpecName: "istio-data") pod "4e09b6ed-f686-4bf8-9aa0-22859f5eed27" (UID: "4e09b6ed-f686-4bf8-9aa0-22859f5eed27"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:56:56.774892 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.774818    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4e09b6ed-f686-4bf8-9aa0-22859f5eed27" (UID: "4e09b6ed-f686-4bf8-9aa0-22859f5eed27"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:56:56.774892 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.774881    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4e09b6ed-f686-4bf8-9aa0-22859f5eed27" (UID: "4e09b6ed-f686-4bf8-9aa0-22859f5eed27"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:56:56.775185 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.775055    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4e09b6ed-f686-4bf8-9aa0-22859f5eed27" (UID: "4e09b6ed-f686-4bf8-9aa0-22859f5eed27"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:56:56.775185 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.775155    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4e09b6ed-f686-4bf8-9aa0-22859f5eed27" (UID: "4e09b6ed-f686-4bf8-9aa0-22859f5eed27"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:56:56.775276 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.775259    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:56:56.775317 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.775275    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:56:56.775317 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.775286    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:56:56.775317 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.775295    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:56:56.775317 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.775303    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:56:56.777426 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.777398    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4e09b6ed-f686-4bf8-9aa0-22859f5eed27" (UID: "4e09b6ed-f686-4bf8-9aa0-22859f5eed27"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:56:56.777544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.777459    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-token" (OuterVolumeSpecName: "istio-token") pod "4e09b6ed-f686-4bf8-9aa0-22859f5eed27" (UID: "4e09b6ed-f686-4bf8-9aa0-22859f5eed27"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:56:56.777544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.777493    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4e09b6ed-f686-4bf8-9aa0-22859f5eed27" (UID: "4e09b6ed-f686-4bf8-9aa0-22859f5eed27"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:56:56.777544 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.777515    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-kube-api-access-wskfn" (OuterVolumeSpecName: "kube-api-access-wskfn") pod "4e09b6ed-f686-4bf8-9aa0-22859f5eed27" (UID: "4e09b6ed-f686-4bf8-9aa0-22859f5eed27"). InnerVolumeSpecName "kube-api-access-wskfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:56:56.876497 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.876456    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:56:56.876497 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.876491    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:56:56.876497 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.876500    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:56:56.876497 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:56.876509    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wskfn\" (UniqueName: \"kubernetes.io/projected/4e09b6ed-f686-4bf8-9aa0-22859f5eed27-kube-api-access-wskfn\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:56:57.475550 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:57.475512    2578 generic.go:356] "Generic (PLEG): container finished" podID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" containerID="d66f6b83c1de5a00e482e3b0d524474bbfff3dea5f303df07eff344aefdbc546" exitCode=0
Sep 09 21:56:57.475733 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:57.475562    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"
Sep 09 21:56:57.475733 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:57.475594    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r" event={"ID":"4e09b6ed-f686-4bf8-9aa0-22859f5eed27","Type":"ContainerDied","Data":"d66f6b83c1de5a00e482e3b0d524474bbfff3dea5f303df07eff344aefdbc546"}
Sep 09 21:56:57.475733 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:57.475650    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r" event={"ID":"4e09b6ed-f686-4bf8-9aa0-22859f5eed27","Type":"ContainerDied","Data":"0230f1a20f985e03966990179e2f3350fac5951f035358942c7f0690ea4f94d6"}
Sep 09 21:56:57.475733 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:57.475672    2578 scope.go:117] "RemoveContainer" containerID="d66f6b83c1de5a00e482e3b0d524474bbfff3dea5f303df07eff344aefdbc546"
Sep 09 21:56:57.485660 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:57.485643    2578 scope.go:117] "RemoveContainer" containerID="38ef4080a7abc67761a9e9df95a8691596361073041afcf8604483b169c8f5b9"
Sep 09 21:56:57.495472 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:57.495450    2578 scope.go:117] "RemoveContainer" containerID="d66f6b83c1de5a00e482e3b0d524474bbfff3dea5f303df07eff344aefdbc546"
Sep 09 21:56:57.495785 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:56:57.495764    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d66f6b83c1de5a00e482e3b0d524474bbfff3dea5f303df07eff344aefdbc546\": container with ID starting with d66f6b83c1de5a00e482e3b0d524474bbfff3dea5f303df07eff344aefdbc546 not found: ID does not exist" containerID="d66f6b83c1de5a00e482e3b0d524474bbfff3dea5f303df07eff344aefdbc546"
Sep 09 21:56:57.495843 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:57.495796    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d66f6b83c1de5a00e482e3b0d524474bbfff3dea5f303df07eff344aefdbc546"} err="failed to get container status \"d66f6b83c1de5a00e482e3b0d524474bbfff3dea5f303df07eff344aefdbc546\": rpc error: code = NotFound desc = could not find container \"d66f6b83c1de5a00e482e3b0d524474bbfff3dea5f303df07eff344aefdbc546\": container with ID starting with d66f6b83c1de5a00e482e3b0d524474bbfff3dea5f303df07eff344aefdbc546 not found: ID does not exist"
Sep 09 21:56:57.495843 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:57.495814    2578 scope.go:117] "RemoveContainer" containerID="38ef4080a7abc67761a9e9df95a8691596361073041afcf8604483b169c8f5b9"
Sep 09 21:56:57.496155 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:56:57.496131    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38ef4080a7abc67761a9e9df95a8691596361073041afcf8604483b169c8f5b9\": container with ID starting with 38ef4080a7abc67761a9e9df95a8691596361073041afcf8604483b169c8f5b9 not found: ID does not exist" containerID="38ef4080a7abc67761a9e9df95a8691596361073041afcf8604483b169c8f5b9"
Sep 09 21:56:57.496221 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:57.496163    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38ef4080a7abc67761a9e9df95a8691596361073041afcf8604483b169c8f5b9"} err="failed to get container status \"38ef4080a7abc67761a9e9df95a8691596361073041afcf8604483b169c8f5b9\": rpc error: code = NotFound desc = could not find container \"38ef4080a7abc67761a9e9df95a8691596361073041afcf8604483b169c8f5b9\": container with ID starting with 38ef4080a7abc67761a9e9df95a8691596361073041afcf8604483b169c8f5b9 not found: ID does not exist"
Sep 09 21:56:57.496865 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:57.496849    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"]
Sep 09 21:56:57.503172 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:57.503149    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4883373b-openshift-default-56585b57fb-k2q5r"]
Sep 09 21:56:58.102367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.102337    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"]
Sep 09 21:56:58.102813 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.102796    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" containerName="istio-proxy"
Sep 09 21:56:58.102857 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.102814    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" containerName="istio-proxy"
Sep 09 21:56:58.102857 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.102823    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9a6d61e-022e-4117-8299-53981a03c554" containerName="maas-api"
Sep 09 21:56:58.102857 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.102829    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9a6d61e-022e-4117-8299-53981a03c554" containerName="maas-api"
Sep 09 21:56:58.102857 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.102840    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" containerName="istio-proxy"
Sep 09 21:56:58.102857 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.102846    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" containerName="istio-proxy"
Sep 09 21:56:58.103058 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.102913    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9a6d61e-022e-4117-8299-53981a03c554" containerName="maas-api"
Sep 09 21:56:58.103058 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.102922    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" containerName="istio-proxy"
Sep 09 21:56:58.103058 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.102927    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" containerName="istio-proxy"
Sep 09 21:56:58.107062 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.107043    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.109665 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.109639    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3fa986\"/\"embedding-w4-3fa986-kserve-self-signed-certs\""
Sep 09 21:56:58.118737 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.118711    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"]
Sep 09 21:56:58.191535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.191497    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-home\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.191709 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.191557    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-dshm\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.191709 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.191596    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds7kb\" (UniqueName: \"kubernetes.io/projected/fd921041-9f42-4bc7-945c-50cb1f428df5-kube-api-access-ds7kb\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.191709 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.191677    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-model-cache\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.191709 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.191699    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd921041-9f42-4bc7-945c-50cb1f428df5-tls-certs\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.191880 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.191756    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-tmp-dir\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.293111 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.293078    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-home\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.293288 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.293136    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-dshm\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.293288 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.293168    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds7kb\" (UniqueName: \"kubernetes.io/projected/fd921041-9f42-4bc7-945c-50cb1f428df5-kube-api-access-ds7kb\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.293288 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.293254    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-model-cache\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.293288 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.293277    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd921041-9f42-4bc7-945c-50cb1f428df5-tls-certs\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.293444 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.293356    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-tmp-dir\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.293553 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.293532    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-home\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.293648 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.293625    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-model-cache\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.293692 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.293635    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-tmp-dir\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.295495 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.295464    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-dshm\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.295721 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.295703    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd921041-9f42-4bc7-945c-50cb1f428df5-tls-certs\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.301067 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.301045    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds7kb\" (UniqueName: \"kubernetes.io/projected/fd921041-9f42-4bc7-945c-50cb1f428df5-kube-api-access-ds7kb\") pod \"embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") " pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.417384 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.417291    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:56:58.697445 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:58.697410    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"]
Sep 09 21:56:58.699103 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:56:58.699076    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd921041_9f42_4bc7_945c_50cb1f428df5.slice/crio-4af26026cb08cd08a5a8e902d2bfb8928a73f2f1744b03d442f21bc3a864969b WatchSource:0}: Error finding container 4af26026cb08cd08a5a8e902d2bfb8928a73f2f1744b03d442f21bc3a864969b: Status 404 returned error can't find the container with id 4af26026cb08cd08a5a8e902d2bfb8928a73f2f1744b03d442f21bc3a864969b
Sep 09 21:56:59.005238 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:59.005160    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e09b6ed-f686-4bf8-9aa0-22859f5eed27" path="/var/lib/kubelet/pods/4e09b6ed-f686-4bf8-9aa0-22859f5eed27/volumes"
Sep 09 21:56:59.495278 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:59.495241    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c" event={"ID":"fd921041-9f42-4bc7-945c-50cb1f428df5","Type":"ContainerStarted","Data":"2bd9410a00b5d1ccf7e1c514297157f7718e55149defb76d39251329fab30e07"}
Sep 09 21:56:59.495278 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:59.495289    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c" event={"ID":"fd921041-9f42-4bc7-945c-50cb1f428df5","Type":"ContainerStarted","Data":"4af26026cb08cd08a5a8e902d2bfb8928a73f2f1744b03d442f21bc3a864969b"}
Sep 09 21:56:59.520323 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:56:59.520278    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c" podStartSLOduration=1.5202631439999998 podStartE2EDuration="1.520263144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:56:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:56:59.5176718 +0000 UTC m=+1438.990636709" watchObservedRunningTime="2026-09-09 21:56:59.520263144 +0000 UTC m=+1438.993228054"
Sep 09 21:57:08.115225 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.115196    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"]
Sep 09 21:57:08.118789 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.118772    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.127724 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.127698    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-390abc\"/\"kube-root-ca.crt\""
Sep 09 21:57:08.128061 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.128037    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-390abc\"/\"default-dockercfg-pszs8\""
Sep 09 21:57:08.128265 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.127785    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-390abc\"/\"embedding-w0-390abc-kserve-self-signed-certs\""
Sep 09 21:57:08.128343 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.127870    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-390abc\"/\"openshift-service-ca.crt\""
Sep 09 21:57:08.133367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.133346    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"]
Sep 09 21:57:08.191734 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.191708    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-tmp-dir\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.191868 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.191772    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/110b9070-ec01-43e4-9985-626dbb6ce436-tls-certs\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.191868 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.191795    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnmzq\" (UniqueName: \"kubernetes.io/projected/110b9070-ec01-43e4-9985-626dbb6ce436-kube-api-access-cnmzq\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.191868 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.191828    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-model-cache\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.192025 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.191925    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-home\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.192025 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.192009    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-dshm\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.293191 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.293148    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/110b9070-ec01-43e4-9985-626dbb6ce436-tls-certs\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.293191 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.293192    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnmzq\" (UniqueName: \"kubernetes.io/projected/110b9070-ec01-43e4-9985-626dbb6ce436-kube-api-access-cnmzq\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.293461 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.293369    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-model-cache\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.293533 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.293482    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-home\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.293600 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.293575    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-dshm\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.293657 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.293645    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-tmp-dir\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.293797 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.293776    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-model-cache\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.293836 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.293810    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-home\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.293927 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.293913    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-tmp-dir\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.295650 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.295629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-dshm\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.295759 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.295727    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/110b9070-ec01-43e4-9985-626dbb6ce436-tls-certs\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.300818 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.300796    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnmzq\" (UniqueName: \"kubernetes.io/projected/110b9070-ec01-43e4-9985-626dbb6ce436-kube-api-access-cnmzq\") pod \"embedding-w0-390abc-kserve-64dd88fdb8-6dmdb\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") " pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.417576 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.417475    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:57:08.417576 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.417534    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:57:08.430436 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.430414    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:57:08.438181 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.438160    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:08.564167 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.564145    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 21:57:08.714646 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:08.714623    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"]
Sep 09 21:57:08.715744 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:57:08.715715    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod110b9070_ec01_43e4_9985_626dbb6ce436.slice/crio-893c9974f866e91fdc7d0380320cade6171a582e1be46f8fc6ebec4438158661 WatchSource:0}: Error finding container 893c9974f866e91fdc7d0380320cade6171a582e1be46f8fc6ebec4438158661: Status 404 returned error can't find the container with id 893c9974f866e91fdc7d0380320cade6171a582e1be46f8fc6ebec4438158661
Sep 09 21:57:09.561252 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:09.561217    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb" event={"ID":"110b9070-ec01-43e4-9985-626dbb6ce436","Type":"ContainerStarted","Data":"95765ca0bd2087bbc49f3f34b006a6c5541da9af9087d6fabcc9de4293b82f4b"}
Sep 09 21:57:09.561669 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:09.561263    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb" event={"ID":"110b9070-ec01-43e4-9985-626dbb6ce436","Type":"ContainerStarted","Data":"893c9974f866e91fdc7d0380320cade6171a582e1be46f8fc6ebec4438158661"}
Sep 09 21:57:09.589578 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:09.589533    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb" podStartSLOduration=1.589519406 podStartE2EDuration="1.589519406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:57:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:57:09.58700324 +0000 UTC m=+1449.059968150" watchObservedRunningTime="2026-09-09 21:57:09.589519406 +0000 UTC m=+1449.062484316"
Sep 09 21:57:18.438896 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:18.438863    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:18.439361 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:18.438911    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:18.452042 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:18.452015    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:18.630451 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:18.630424    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:57:26.699307 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:26.699274    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:57:46.772960 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:46.772930    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"]
Sep 09 21:57:46.776805 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:46.776788    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:46.779211 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:46.779188    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-5d8401-openshift-default-dockercfg-jpqfl\""
Sep 09 21:57:46.788186 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:46.788161    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"]
Sep 09 21:57:46.966505 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:46.966473    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-envoy\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:46.966679 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:46.966530    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c7834d5-06cf-4316-b19a-4613bb375eb1-istiod-ca-cert\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:46.966679 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:46.966578    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-workload-socket\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:46.966679 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:46.966599    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-token\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:46.966679 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:46.966663    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z56lp\" (UniqueName: \"kubernetes.io/projected/4c7834d5-06cf-4316-b19a-4613bb375eb1-kube-api-access-z56lp\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:46.966679 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:46.966679    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-workload-certs\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:46.966860 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:46.966697    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-credential-socket\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:46.966860 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:46.966742    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-podinfo\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:46.966860 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:46.966789    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-data\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.067620 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.067537    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-envoy\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.067620 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.067593    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c7834d5-06cf-4316-b19a-4613bb375eb1-istiod-ca-cert\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.067848 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.067631    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-workload-socket\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.067848 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.067652    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-token\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.067848 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.067679    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z56lp\" (UniqueName: \"kubernetes.io/projected/4c7834d5-06cf-4316-b19a-4613bb375eb1-kube-api-access-z56lp\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.067848 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.067696    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-workload-certs\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.067848 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.067792    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-credential-socket\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.068171 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.068070    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-workload-certs\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.068171 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.068131    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-workload-socket\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.068171 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.068140    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-podinfo\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.068335 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.068187    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-data\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.068335 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.068228    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-credential-socket\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.068487 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.068455    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c7834d5-06cf-4316-b19a-4613bb375eb1-istiod-ca-cert\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.068603 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.068511    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-data\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.070012 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.069986    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-envoy\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.070412 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.070394    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-podinfo\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.077578 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.077546    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-token\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.078843 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.078819    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z56lp\" (UniqueName: \"kubernetes.io/projected/4c7834d5-06cf-4316-b19a-4613bb375eb1-kube-api-access-z56lp\") pod \"e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") " pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.087642 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.087623    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:47.375106 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.375071    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"]
Sep 09 21:57:47.375263 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:57:47.375201    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c7834d5_06cf_4316_b19a_4613bb375eb1.slice/crio-45f73579a0f00b44346d07550a2c5096637c303bbb4f063dad04795f5e6c8624 WatchSource:0}: Error finding container 45f73579a0f00b44346d07550a2c5096637c303bbb4f063dad04795f5e6c8624: Status 404 returned error can't find the container with id 45f73579a0f00b44346d07550a2c5096637c303bbb4f063dad04795f5e6c8624
Sep 09 21:57:47.377619 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.377582    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:57:47.377739 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.377674    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:57:47.377739 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.377717    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:57:47.810167 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.810131    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4" event={"ID":"4c7834d5-06cf-4316-b19a-4613bb375eb1","Type":"ContainerStarted","Data":"941efd20e2e1e30fdd10f3a32866c0453cfa30c6bda2fbb436670d993bf0cf64"}
Sep 09 21:57:47.810554 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.810193    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4" event={"ID":"4c7834d5-06cf-4316-b19a-4613bb375eb1","Type":"ContainerStarted","Data":"45f73579a0f00b44346d07550a2c5096637c303bbb4f063dad04795f5e6c8624"}
Sep 09 21:57:47.837682 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:47.837631    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4" podStartSLOduration=1.837612165 podStartE2EDuration="1.837612165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:57:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:57:47.834733016 +0000 UTC m=+1487.307697929" watchObservedRunningTime="2026-09-09 21:57:47.837612165 +0000 UTC m=+1487.310577077"
Sep 09 21:57:48.088781 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:48.088696    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:48.094063 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:48.094037    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:48.816665 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:48.816627    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:48.817535 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:48.817516    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:57:50.947463 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:50.947433    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"]
Sep 09 21:57:50.951584 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:50.951564    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:57:50.955446 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:50.955423    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-5d8401-serving-cert\""
Sep 09 21:57:50.965464 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:50.965444    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"]
Sep 09 21:57:51.007018 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.006994    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-maas-api-tls\") pod \"maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:57:51.007162 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.007090    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-metrics-certs\") pod \"maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:57:51.007162 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.007126    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-openshift-service-ca\") pod \"maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:57:51.007259 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.007176    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bxcc\" (UniqueName: \"kubernetes.io/projected/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-kube-api-access-8bxcc\") pod \"maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:57:51.107767 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.107734    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-metrics-certs\") pod \"maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:57:51.107933 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.107782    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-openshift-service-ca\") pod \"maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:57:51.107933 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.107830    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bxcc\" (UniqueName: \"kubernetes.io/projected/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-kube-api-access-8bxcc\") pod \"maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:57:51.108094 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.107932    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-maas-api-tls\") pod \"maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:57:51.108554 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.108522    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-openshift-service-ca\") pod \"maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:57:51.110299 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.110277    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-metrics-certs\") pod \"maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:57:51.110394 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.110332    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-maas-api-tls\") pod \"maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:57:51.118324 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.118295    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bxcc\" (UniqueName: \"kubernetes.io/projected/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-kube-api-access-8bxcc\") pod \"maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:57:51.262392 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.262313    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:57:51.546387 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.546354    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"]
Sep 09 21:57:51.554710 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:57:51.554681    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ea1bbd_d8d2_4b8a_9dee_1fe10ea6e99b.slice/crio-4ef45a4847263d7a717d1099ce01aba973d71df9244dbab862ed687956a8e58a WatchSource:0}: Error finding container 4ef45a4847263d7a717d1099ce01aba973d71df9244dbab862ed687956a8e58a: Status 404 returned error can't find the container with id 4ef45a4847263d7a717d1099ce01aba973d71df9244dbab862ed687956a8e58a
Sep 09 21:57:51.556314 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.556295    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 21:57:51.840891 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:51.840799    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8" event={"ID":"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b","Type":"ContainerStarted","Data":"4ef45a4847263d7a717d1099ce01aba973d71df9244dbab862ed687956a8e58a"}
Sep 09 21:57:52.848695 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:52.848657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8" event={"ID":"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b","Type":"ContainerStarted","Data":"8f58a5008a9e1bd4092d0579f4d8e6ea9bc8ef8c8d0ab82b126d3c2c1a662ecd"}
Sep 09 21:57:52.849102 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:52.848758    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:57:52.868837 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:52.868785    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8" podStartSLOduration=2.174959628 podStartE2EDuration="2.86877128s" totalImagesPullingTime="693.811652ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:57:52.867080067 +0000 UTC m=+1492.340044978" watchObservedRunningTime="2026-09-09 21:57:52.86877128 +0000 UTC m=+1492.341736190"
Sep 09 21:57:58.860321 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:57:58.860290    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:58:01.108591 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:01.108560    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 21:58:01.114172 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:01.114149    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 21:58:01.300587 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:01.300560    2578 scope.go:117] "RemoveContainer" containerID="495071bf5629b75c930c46216d1fbd508d2e094428944a57c47b3f934d9f4b73"
Sep 09 21:58:15.456129 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.456092    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"]
Sep 09 21:58:15.459827 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.459799    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.464253 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.464231    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-9dedad-openshift-default-dockercfg-vsxmx\""
Sep 09 21:58:15.479410 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.479387    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"]
Sep 09 21:58:15.541551 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.540856    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-data\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.541551 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.540912    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-podinfo\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.541551 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.540944    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-workload-certs\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.541551 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.541025    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-workload-socket\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.541551 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.541073    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp8xx\" (UniqueName: \"kubernetes.io/projected/0e0b223d-abfc-42bd-ae44-1c828f2c2892-kube-api-access-xp8xx\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.541551 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.541173    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-token\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.541551 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.541202    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-envoy\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.541551 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.541309    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-credential-socket\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.541551 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.541337    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istiod-ca-cert\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.642902 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.642865    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-token\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.642902 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.642905    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-envoy\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.643156 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.643006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-credential-socket\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.643156 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.643039    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istiod-ca-cert\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.643156 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.643074    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-data\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.643278 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.643258    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-podinfo\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.643322 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.643295    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-workload-certs\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.643371 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.643351    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-workload-socket\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.643427 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.643400    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp8xx\" (UniqueName: \"kubernetes.io/projected/0e0b223d-abfc-42bd-ae44-1c828f2c2892-kube-api-access-xp8xx\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.643507 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.643487    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-data\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.643557 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.643520    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-credential-socket\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.643822 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.643800    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istiod-ca-cert\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.643909 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.643793    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-workload-certs\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.644046 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.644026    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-workload-socket\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.645297 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.645280    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-envoy\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.645707 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.645689    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-podinfo\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.656398 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.656372    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-token\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.657260 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.657237    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp8xx\" (UniqueName: \"kubernetes.io/projected/0e0b223d-abfc-42bd-ae44-1c828f2c2892-kube-api-access-xp8xx\") pod \"e2e-api-b-9dedad-openshift-default-ccb56564-mppvx\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") " pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:15.770409 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:15.770390    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:16.057092 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:16.057069    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"]
Sep 09 21:58:16.059749 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:58:16.059724    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e0b223d_abfc_42bd_ae44_1c828f2c2892.slice/crio-8604296b7e08b9ee9bbfb8715542fecbb9470bbbe703744fdd23cfbfd58a7225 WatchSource:0}: Error finding container 8604296b7e08b9ee9bbfb8715542fecbb9470bbbe703744fdd23cfbfd58a7225: Status 404 returned error can't find the container with id 8604296b7e08b9ee9bbfb8715542fecbb9470bbbe703744fdd23cfbfd58a7225
Sep 09 21:58:16.061952 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:16.061919    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:58:16.062059 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:16.062010    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:58:16.062059 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:16.062040    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:58:17.008153 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:17.008120    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx" event={"ID":"0e0b223d-abfc-42bd-ae44-1c828f2c2892","Type":"ContainerStarted","Data":"0f2ecdd3dbcc1d1fcc2ae06e1445c49b2499cee8ab008f90b2a8f1b4f4595c9d"}
Sep 09 21:58:17.008521 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:17.008170    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx" event={"ID":"0e0b223d-abfc-42bd-ae44-1c828f2c2892","Type":"ContainerStarted","Data":"8604296b7e08b9ee9bbfb8715542fecbb9470bbbe703744fdd23cfbfd58a7225"}
Sep 09 21:58:17.046313 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:17.046265    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx" podStartSLOduration=2.046251241 podStartE2EDuration="2.046251241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:58:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:58:17.045759539 +0000 UTC m=+1516.518724445" watchObservedRunningTime="2026-09-09 21:58:17.046251241 +0000 UTC m=+1516.519216151"
Sep 09 21:58:17.771142 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:17.771105    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:17.776559 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:17.776534    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:18.014861 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:18.014829    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:18.015742 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:18.015725    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:20.293139 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:20.293096    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"]
Sep 09 21:58:20.298018 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:20.297959    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:20.301106 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:20.301083    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-9dedad-serving-cert\""
Sep 09 21:58:20.315809 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:20.315786    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"]
Sep 09 21:58:20.394712 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:20.394684    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a540ba2-b7b7-466a-aa3b-16170599fdbd-maas-api-tls\") pod \"maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:20.394860 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:20.394751    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a540ba2-b7b7-466a-aa3b-16170599fdbd-openshift-service-ca\") pod \"maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:20.394860 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:20.394795    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a540ba2-b7b7-466a-aa3b-16170599fdbd-metrics-certs\") pod \"maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:20.394953 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:20.394877    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t9b2\" (UniqueName: \"kubernetes.io/projected/7a540ba2-b7b7-466a-aa3b-16170599fdbd-kube-api-access-6t9b2\") pod \"maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:20.495805 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:20.495758    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a540ba2-b7b7-466a-aa3b-16170599fdbd-openshift-service-ca\") pod \"maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:20.496042 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:20.495846    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a540ba2-b7b7-466a-aa3b-16170599fdbd-metrics-certs\") pod \"maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:20.496042 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:20.495921    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t9b2\" (UniqueName: \"kubernetes.io/projected/7a540ba2-b7b7-466a-aa3b-16170599fdbd-kube-api-access-6t9b2\") pod \"maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:20.496042 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:20.496026    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a540ba2-b7b7-466a-aa3b-16170599fdbd-maas-api-tls\") pod \"maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:20.496206 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:58:20.496146    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-serving-cert: secret "maas-api-e2e-api-b-9dedad-serving-cert" not found
Sep 09 21:58:20.496255 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:58:20.496242    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a540ba2-b7b7-466a-aa3b-16170599fdbd-maas-api-tls podName:7a540ba2-b7b7-466a-aa3b-16170599fdbd nodeName:}" failed. No retries permitted until 2026-09-09 21:58:20.996217158 +0000 UTC m=+1520.469182062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7a540ba2-b7b7-466a-aa3b-16170599fdbd-maas-api-tls") pod "maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j" (UID: "7a540ba2-b7b7-466a-aa3b-16170599fdbd") : secret "maas-api-e2e-api-b-9dedad-serving-cert" not found
Sep 09 21:58:20.496533 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:20.496512    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a540ba2-b7b7-466a-aa3b-16170599fdbd-openshift-service-ca\") pod \"maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:20.498447 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:20.498424    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a540ba2-b7b7-466a-aa3b-16170599fdbd-metrics-certs\") pod \"maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:20.507474 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:20.507452    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t9b2\" (UniqueName: \"kubernetes.io/projected/7a540ba2-b7b7-466a-aa3b-16170599fdbd-kube-api-access-6t9b2\") pod \"maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:21.001613 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:21.001581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a540ba2-b7b7-466a-aa3b-16170599fdbd-maas-api-tls\") pod \"maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:21.004146 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:21.004118    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a540ba2-b7b7-466a-aa3b-16170599fdbd-maas-api-tls\") pod \"maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:21.208716 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:21.208683    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:21.484803 ip-10-0-140-157 kubenswrapper[2578]: W0909 21:58:21.484778    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a540ba2_b7b7_466a_aa3b_16170599fdbd.slice/crio-a8ca5612d0b1d9e23f1e2667945581903e4607765d841cf648f2ec5eda80d7dd WatchSource:0}: Error finding container a8ca5612d0b1d9e23f1e2667945581903e4607765d841cf648f2ec5eda80d7dd: Status 404 returned error can't find the container with id a8ca5612d0b1d9e23f1e2667945581903e4607765d841cf648f2ec5eda80d7dd
Sep 09 21:58:21.485215 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:21.485045    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"]
Sep 09 21:58:22.052808 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:22.052723    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j" event={"ID":"7a540ba2-b7b7-466a-aa3b-16170599fdbd","Type":"ContainerStarted","Data":"8e409b058cdd38c73d7870ea9dfd90d2f1634312787515bd1b237b9beb43c39a"}
Sep 09 21:58:22.052945 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:22.052881    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:22.052945 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:22.052902    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j" event={"ID":"7a540ba2-b7b7-466a-aa3b-16170599fdbd","Type":"ContainerStarted","Data":"a8ca5612d0b1d9e23f1e2667945581903e4607765d841cf648f2ec5eda80d7dd"}
Sep 09 21:58:22.073659 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:22.073596    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j" podStartSLOduration=1.765546308 podStartE2EDuration="2.07358244s" totalImagesPullingTime="308.036132ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:58:22.071204541 +0000 UTC m=+1521.544169451" watchObservedRunningTime="2026-09-09 21:58:22.07358244 +0000 UTC m=+1521.546547350"
Sep 09 21:58:28.064588 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:28.064514    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:58:32.515410 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:32.515371    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:58:45.228995 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:45.228939    2578 generic.go:356] "Generic (PLEG): container finished" podID="4c7834d5-06cf-4316-b19a-4613bb375eb1" containerID="941efd20e2e1e30fdd10f3a32866c0453cfa30c6bda2fbb436670d993bf0cf64" exitCode=0
Sep 09 21:58:45.229434 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:45.229006    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4" event={"ID":"4c7834d5-06cf-4316-b19a-4613bb375eb1","Type":"ContainerDied","Data":"941efd20e2e1e30fdd10f3a32866c0453cfa30c6bda2fbb436670d993bf0cf64"}
Sep 09 21:58:45.229434 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:45.229420    2578 scope.go:117] "RemoveContainer" containerID="941efd20e2e1e30fdd10f3a32866c0453cfa30c6bda2fbb436670d993bf0cf64"
Sep 09 21:58:45.230787 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:45.230765    2578 generic.go:356] "Generic (PLEG): container finished" podID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" containerID="0f2ecdd3dbcc1d1fcc2ae06e1445c49b2499cee8ab008f90b2a8f1b4f4595c9d" exitCode=0
Sep 09 21:58:45.230891 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:45.230782    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:58:45.230891 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:45.230840    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx" event={"ID":"0e0b223d-abfc-42bd-ae44-1c828f2c2892","Type":"ContainerDied","Data":"0f2ecdd3dbcc1d1fcc2ae06e1445c49b2499cee8ab008f90b2a8f1b4f4595c9d"}
Sep 09 21:58:45.231061 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:45.230847    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:58:45.231061 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:45.230985    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:58:45.231302 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:45.231281    2578 scope.go:117] "RemoveContainer" containerID="0f2ecdd3dbcc1d1fcc2ae06e1445c49b2499cee8ab008f90b2a8f1b4f4595c9d"
Sep 09 21:58:45.232426 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:45.232402    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:58:45.232496 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:45.232461    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:58:45.232496 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:45.232491    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 21:58:46.249272 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:46.249224    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4" event={"ID":"4c7834d5-06cf-4316-b19a-4613bb375eb1","Type":"ContainerStarted","Data":"2a004b6a401ddeff62900630a4dbc3afae1b0d37a944e0fd668b1bfeb2ce9110"}
Sep 09 21:58:46.253593 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:46.253551    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx" event={"ID":"0e0b223d-abfc-42bd-ae44-1c828f2c2892","Type":"ContainerStarted","Data":"1b394c0404c3bf5c00835aed7cf3a7d086f0413b150712d7edd107471b4c250b"}
Sep 09 21:58:46.772911 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:46.772865    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:46.773176 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:46.773032    2578 patch_prober.go:28] interesting pod/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Sep 09 21:58:46.773176 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:46.773081    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx" podUID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Sep 09 21:58:47.088742 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:47.088648    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:58:47.089528 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:47.089486    2578 patch_prober.go:28] interesting pod/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Sep 09 21:58:47.089650 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:47.089572    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4" podUID="4c7834d5-06cf-4316-b19a-4613bb375eb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Sep 09 21:58:47.771324 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:47.771282    2578 patch_prober.go:28] interesting pod/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Sep 09 21:58:47.771813 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:47.771365    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx" podUID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Sep 09 21:58:48.015657 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:48.015625    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:48.088321 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:48.088229    2578 patch_prober.go:28] interesting pod/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Sep 09 21:58:48.088321 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:48.088284    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4" podUID="4c7834d5-06cf-4316-b19a-4613bb375eb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Sep 09 21:58:48.517983 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:48.517949    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"]
Sep 09 21:58:48.771027 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:48.770906    2578 patch_prober.go:28] interesting pod/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Sep 09 21:58:48.771027 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:48.770994    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx" podUID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Sep 09 21:58:48.817228 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:48.817169    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:58:49.089047 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:49.088483    2578 patch_prober.go:28] interesting pod/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Sep 09 21:58:49.089047 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:49.088533    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4" podUID="4c7834d5-06cf-4316-b19a-4613bb375eb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Sep 09 21:58:49.771085 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:49.771040    2578 patch_prober.go:28] interesting pod/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Sep 09 21:58:49.771272 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:49.771110    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx" podUID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Sep 09 21:58:50.088310 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:50.088211    2578 patch_prober.go:28] interesting pod/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Sep 09 21:58:50.088310 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:50.088272    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4" podUID="4c7834d5-06cf-4316-b19a-4613bb375eb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Sep 09 21:58:50.772328 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:50.771815    2578 patch_prober.go:28] interesting pod/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Sep 09 21:58:50.772328 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:50.771885    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx" podUID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Sep 09 21:58:51.089135 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:51.089052    2578 patch_prober.go:28] interesting pod/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Sep 09 21:58:51.089135 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:51.089111    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4" podUID="4c7834d5-06cf-4316-b19a-4613bb375eb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Sep 09 21:58:51.774579 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:51.774548    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:51.775594 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:51.775572    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:58:52.092141 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:52.092059    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:58:52.093053 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:58:52.093035    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:59:03.008738 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.008707    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"]
Sep 09 21:59:03.009240 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.008885    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8" podUID="73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b" containerName="maas-api" containerID="cri-o://8f58a5008a9e1bd4092d0579f4d8e6ea9bc8ef8c8d0ab82b126d3c2c1a662ecd" gracePeriod=30
Sep 09 21:59:03.312356 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.312058    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:59:03.379374 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.379338    2578 generic.go:356] "Generic (PLEG): container finished" podID="73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b" containerID="8f58a5008a9e1bd4092d0579f4d8e6ea9bc8ef8c8d0ab82b126d3c2c1a662ecd" exitCode=0
Sep 09 21:59:03.379374 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.379376    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"
Sep 09 21:59:03.379581 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.379377    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8" event={"ID":"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b","Type":"ContainerDied","Data":"8f58a5008a9e1bd4092d0579f4d8e6ea9bc8ef8c8d0ab82b126d3c2c1a662ecd"}
Sep 09 21:59:03.379581 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.379504    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8" event={"ID":"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b","Type":"ContainerDied","Data":"4ef45a4847263d7a717d1099ce01aba973d71df9244dbab862ed687956a8e58a"}
Sep 09 21:59:03.379581 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.379535    2578 scope.go:117] "RemoveContainer" containerID="8f58a5008a9e1bd4092d0579f4d8e6ea9bc8ef8c8d0ab82b126d3c2c1a662ecd"
Sep 09 21:59:03.392571 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.392550    2578 scope.go:117] "RemoveContainer" containerID="8f58a5008a9e1bd4092d0579f4d8e6ea9bc8ef8c8d0ab82b126d3c2c1a662ecd"
Sep 09 21:59:03.392904 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:59:03.392875    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f58a5008a9e1bd4092d0579f4d8e6ea9bc8ef8c8d0ab82b126d3c2c1a662ecd\": container with ID starting with 8f58a5008a9e1bd4092d0579f4d8e6ea9bc8ef8c8d0ab82b126d3c2c1a662ecd not found: ID does not exist" containerID="8f58a5008a9e1bd4092d0579f4d8e6ea9bc8ef8c8d0ab82b126d3c2c1a662ecd"
Sep 09 21:59:03.393031 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.392916    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f58a5008a9e1bd4092d0579f4d8e6ea9bc8ef8c8d0ab82b126d3c2c1a662ecd"} err="failed to get container status \"8f58a5008a9e1bd4092d0579f4d8e6ea9bc8ef8c8d0ab82b126d3c2c1a662ecd\": rpc error: code = NotFound desc = could not find container \"8f58a5008a9e1bd4092d0579f4d8e6ea9bc8ef8c8d0ab82b126d3c2c1a662ecd\": container with ID starting with 8f58a5008a9e1bd4092d0579f4d8e6ea9bc8ef8c8d0ab82b126d3c2c1a662ecd not found: ID does not exist"
Sep 09 21:59:03.484410 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.484374    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-maas-api-tls\") pod \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") "
Sep 09 21:59:03.484410 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.484424    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-metrics-certs\") pod \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") "
Sep 09 21:59:03.484693 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.484490    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bxcc\" (UniqueName: \"kubernetes.io/projected/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-kube-api-access-8bxcc\") pod \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") "
Sep 09 21:59:03.484693 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.484546    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-openshift-service-ca\") pod \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\" (UID: \"73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b\") "
Sep 09 21:59:03.484955 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.484929    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b" (UID: "73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:59:03.486610 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.486581    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b" (UID: "73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:59:03.486712 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.486615    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b" (UID: "73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:59:03.486712 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.486637    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-kube-api-access-8bxcc" (OuterVolumeSpecName: "kube-api-access-8bxcc") pod "73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b" (UID: "73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b"). InnerVolumeSpecName "kube-api-access-8bxcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:59:03.586385 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.586327    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-maas-api-tls\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:03.586385 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.586350    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-metrics-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:03.586385 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.586366    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bxcc\" (UniqueName: \"kubernetes.io/projected/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-kube-api-access-8bxcc\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:03.586385 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.586377    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b-openshift-service-ca\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:03.590525 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.590495    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb" podUID="110b9070-ec01-43e4-9985-626dbb6ce436" containerName="main" containerID="cri-o://95765ca0bd2087bbc49f3f34b006a6c5541da9af9087d6fabcc9de4293b82f4b" gracePeriod=45
Sep 09 21:59:03.702173 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.702151    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"]
Sep 09 21:59:03.706825 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.706723    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5d8401-779c64fcf9-sg9v8"]
Sep 09 21:59:03.817156 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.817135    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:59:03.989915 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.989831    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnmzq\" (UniqueName: \"kubernetes.io/projected/110b9070-ec01-43e4-9985-626dbb6ce436-kube-api-access-cnmzq\") pod \"110b9070-ec01-43e4-9985-626dbb6ce436\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") "
Sep 09 21:59:03.990113 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.989921    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-model-cache\") pod \"110b9070-ec01-43e4-9985-626dbb6ce436\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") "
Sep 09 21:59:03.990113 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.990074    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-dshm\") pod \"110b9070-ec01-43e4-9985-626dbb6ce436\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") "
Sep 09 21:59:03.990275 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.990126    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-home\") pod \"110b9070-ec01-43e4-9985-626dbb6ce436\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") "
Sep 09 21:59:03.990946 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.990387    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-model-cache" (OuterVolumeSpecName: "model-cache") pod "110b9070-ec01-43e4-9985-626dbb6ce436" (UID: "110b9070-ec01-43e4-9985-626dbb6ce436"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:03.990946 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.990520    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-tmp-dir\") pod \"110b9070-ec01-43e4-9985-626dbb6ce436\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") "
Sep 09 21:59:03.990946 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.990749    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "110b9070-ec01-43e4-9985-626dbb6ce436" (UID: "110b9070-ec01-43e4-9985-626dbb6ce436"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:03.990946 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.990795    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-home" (OuterVolumeSpecName: "home") pod "110b9070-ec01-43e4-9985-626dbb6ce436" (UID: "110b9070-ec01-43e4-9985-626dbb6ce436"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:03.990946 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.990869    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/110b9070-ec01-43e4-9985-626dbb6ce436-tls-certs\") pod \"110b9070-ec01-43e4-9985-626dbb6ce436\" (UID: \"110b9070-ec01-43e4-9985-626dbb6ce436\") "
Sep 09 21:59:03.991347 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.991325    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-home\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:03.991412 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.991352    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-tmp-dir\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:03.991412 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.991367    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-model-cache\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:03.993279 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.993237    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/110b9070-ec01-43e4-9985-626dbb6ce436-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "110b9070-ec01-43e4-9985-626dbb6ce436" (UID: "110b9070-ec01-43e4-9985-626dbb6ce436"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:59:03.993279 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.993264    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-dshm" (OuterVolumeSpecName: "dshm") pod "110b9070-ec01-43e4-9985-626dbb6ce436" (UID: "110b9070-ec01-43e4-9985-626dbb6ce436"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:03.993500 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:03.993473    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/110b9070-ec01-43e4-9985-626dbb6ce436-kube-api-access-cnmzq" (OuterVolumeSpecName: "kube-api-access-cnmzq") pod "110b9070-ec01-43e4-9985-626dbb6ce436" (UID: "110b9070-ec01-43e4-9985-626dbb6ce436"). InnerVolumeSpecName "kube-api-access-cnmzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:59:04.092177 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:04.092147    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/110b9070-ec01-43e4-9985-626dbb6ce436-tls-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:04.092177 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:04.092176    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnmzq\" (UniqueName: \"kubernetes.io/projected/110b9070-ec01-43e4-9985-626dbb6ce436-kube-api-access-cnmzq\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:04.092541 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:04.092186    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/110b9070-ec01-43e4-9985-626dbb6ce436-dshm\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:04.386472 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:04.386441    2578 generic.go:356] "Generic (PLEG): container finished" podID="110b9070-ec01-43e4-9985-626dbb6ce436" containerID="95765ca0bd2087bbc49f3f34b006a6c5541da9af9087d6fabcc9de4293b82f4b" exitCode=0
Sep 09 21:59:04.386659 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:04.386486    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"
Sep 09 21:59:04.386659 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:04.386522    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb" event={"ID":"110b9070-ec01-43e4-9985-626dbb6ce436","Type":"ContainerDied","Data":"95765ca0bd2087bbc49f3f34b006a6c5541da9af9087d6fabcc9de4293b82f4b"}
Sep 09 21:59:04.386659 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:04.386570    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb" event={"ID":"110b9070-ec01-43e4-9985-626dbb6ce436","Type":"ContainerDied","Data":"893c9974f866e91fdc7d0380320cade6171a582e1be46f8fc6ebec4438158661"}
Sep 09 21:59:04.386659 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:04.386590    2578 scope.go:117] "RemoveContainer" containerID="95765ca0bd2087bbc49f3f34b006a6c5541da9af9087d6fabcc9de4293b82f4b"
Sep 09 21:59:04.397043 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:04.397026    2578 scope.go:117] "RemoveContainer" containerID="95765ca0bd2087bbc49f3f34b006a6c5541da9af9087d6fabcc9de4293b82f4b"
Sep 09 21:59:04.397320 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:59:04.397299    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95765ca0bd2087bbc49f3f34b006a6c5541da9af9087d6fabcc9de4293b82f4b\": container with ID starting with 95765ca0bd2087bbc49f3f34b006a6c5541da9af9087d6fabcc9de4293b82f4b not found: ID does not exist" containerID="95765ca0bd2087bbc49f3f34b006a6c5541da9af9087d6fabcc9de4293b82f4b"
Sep 09 21:59:04.397419 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:04.397327    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95765ca0bd2087bbc49f3f34b006a6c5541da9af9087d6fabcc9de4293b82f4b"} err="failed to get container status \"95765ca0bd2087bbc49f3f34b006a6c5541da9af9087d6fabcc9de4293b82f4b\": rpc error: code = NotFound desc = could not find container \"95765ca0bd2087bbc49f3f34b006a6c5541da9af9087d6fabcc9de4293b82f4b\": container with ID starting with 95765ca0bd2087bbc49f3f34b006a6c5541da9af9087d6fabcc9de4293b82f4b not found: ID does not exist"
Sep 09 21:59:04.409738 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:04.409709    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"]
Sep 09 21:59:04.415846 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:04.415824    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-390abc/embedding-w0-390abc-kserve-64dd88fdb8-6dmdb"]
Sep 09 21:59:05.004138 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:05.004107    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="110b9070-ec01-43e4-9985-626dbb6ce436" path="/var/lib/kubelet/pods/110b9070-ec01-43e4-9985-626dbb6ce436/volumes"
Sep 09 21:59:05.004515 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:05.004502    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b" path="/var/lib/kubelet/pods/73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b/volumes"
Sep 09 21:59:06.871373 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:06.871337    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"]
Sep 09 21:59:06.871754 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:06.871550    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb" podUID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerName="istio-proxy" containerID="cri-o://16475e0352609543452b124a4979663dd977f88b3bba7bebcaaa9378b7136ac7" gracePeriod=30
Sep 09 21:59:09.812460 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:09.812428    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"]
Sep 09 21:59:09.812928 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:09.812628    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4" podUID="4c7834d5-06cf-4316-b19a-4613bb375eb1" containerName="istio-proxy" containerID="cri-o://2a004b6a401ddeff62900630a4dbc3afae1b0d37a944e0fd668b1bfeb2ce9110" gracePeriod=30
Sep 09 21:59:12.134428 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.134371    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:59:12.168108 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.168079    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-workload-certs\") pod \"06282b23-1694-4fc2-b3c3-c17e924cca00\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") "
Sep 09 21:59:12.168233 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.168113    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-data\") pod \"06282b23-1694-4fc2-b3c3-c17e924cca00\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") "
Sep 09 21:59:12.168233 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.168156    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-workload-socket\") pod \"06282b23-1694-4fc2-b3c3-c17e924cca00\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") "
Sep 09 21:59:12.168349 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.168256    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-credential-socket\") pod \"06282b23-1694-4fc2-b3c3-c17e924cca00\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") "
Sep 09 21:59:12.168349 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.168297    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-podinfo\") pod \"06282b23-1694-4fc2-b3c3-c17e924cca00\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") "
Sep 09 21:59:12.168456 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.168375    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-envoy\") pod \"06282b23-1694-4fc2-b3c3-c17e924cca00\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") "
Sep 09 21:59:12.168510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.168340    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "06282b23-1694-4fc2-b3c3-c17e924cca00" (UID: "06282b23-1694-4fc2-b3c3-c17e924cca00"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:12.168510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.168413    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-data" (OuterVolumeSpecName: "istio-data") pod "06282b23-1694-4fc2-b3c3-c17e924cca00" (UID: "06282b23-1694-4fc2-b3c3-c17e924cca00"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:12.168510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.168482    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "06282b23-1694-4fc2-b3c3-c17e924cca00" (UID: "06282b23-1694-4fc2-b3c3-c17e924cca00"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:12.168510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.168468    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06282b23-1694-4fc2-b3c3-c17e924cca00-istiod-ca-cert\") pod \"06282b23-1694-4fc2-b3c3-c17e924cca00\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") "
Sep 09 21:59:12.168732 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.168510    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "06282b23-1694-4fc2-b3c3-c17e924cca00" (UID: "06282b23-1694-4fc2-b3c3-c17e924cca00"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:12.168732 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.168548    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97hrc\" (UniqueName: \"kubernetes.io/projected/06282b23-1694-4fc2-b3c3-c17e924cca00-kube-api-access-97hrc\") pod \"06282b23-1694-4fc2-b3c3-c17e924cca00\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") "
Sep 09 21:59:12.168732 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.168576    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-token\") pod \"06282b23-1694-4fc2-b3c3-c17e924cca00\" (UID: \"06282b23-1694-4fc2-b3c3-c17e924cca00\") "
Sep 09 21:59:12.169153 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.169055    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:12.169153 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.169080    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:12.169153 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.169093    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:12.169153 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.169107    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:12.169519 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.169171    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06282b23-1694-4fc2-b3c3-c17e924cca00-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "06282b23-1694-4fc2-b3c3-c17e924cca00" (UID: "06282b23-1694-4fc2-b3c3-c17e924cca00"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:59:12.171041 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.170997    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06282b23-1694-4fc2-b3c3-c17e924cca00-kube-api-access-97hrc" (OuterVolumeSpecName: "kube-api-access-97hrc") pod "06282b23-1694-4fc2-b3c3-c17e924cca00" (UID: "06282b23-1694-4fc2-b3c3-c17e924cca00"). InnerVolumeSpecName "kube-api-access-97hrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:59:12.171139 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.171096    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "06282b23-1694-4fc2-b3c3-c17e924cca00" (UID: "06282b23-1694-4fc2-b3c3-c17e924cca00"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:12.171139 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.171126    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-token" (OuterVolumeSpecName: "istio-token") pod "06282b23-1694-4fc2-b3c3-c17e924cca00" (UID: "06282b23-1694-4fc2-b3c3-c17e924cca00"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:59:12.171237 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.171167    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "06282b23-1694-4fc2-b3c3-c17e924cca00" (UID: "06282b23-1694-4fc2-b3c3-c17e924cca00"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:59:12.270837 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.270803    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:12.270837 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.270837    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06282b23-1694-4fc2-b3c3-c17e924cca00-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:12.271034 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.270852    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97hrc\" (UniqueName: \"kubernetes.io/projected/06282b23-1694-4fc2-b3c3-c17e924cca00-kube-api-access-97hrc\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:12.271034 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.270868    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:12.271034 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.270881    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06282b23-1694-4fc2-b3c3-c17e924cca00-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:12.439724 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.439640    2578 generic.go:356] "Generic (PLEG): container finished" podID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerID="16475e0352609543452b124a4979663dd977f88b3bba7bebcaaa9378b7136ac7" exitCode=0
Sep 09 21:59:12.439724 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.439693    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"
Sep 09 21:59:12.439898 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.439727    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb" event={"ID":"06282b23-1694-4fc2-b3c3-c17e924cca00","Type":"ContainerDied","Data":"16475e0352609543452b124a4979663dd977f88b3bba7bebcaaa9378b7136ac7"}
Sep 09 21:59:12.439898 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.439804    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb" event={"ID":"06282b23-1694-4fc2-b3c3-c17e924cca00","Type":"ContainerDied","Data":"7ef5e13e3cd15a51fb1548791bb49dccbd2bb4508f06796c9bd096958b665a8e"}
Sep 09 21:59:12.439898 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.439829    2578 scope.go:117] "RemoveContainer" containerID="16475e0352609543452b124a4979663dd977f88b3bba7bebcaaa9378b7136ac7"
Sep 09 21:59:12.450192 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.450173    2578 scope.go:117] "RemoveContainer" containerID="af1f598aba0235e89dd793bb0ff0bca71e507ca350c7804a4a7d464bc1a2f955"
Sep 09 21:59:12.458953 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.458933    2578 scope.go:117] "RemoveContainer" containerID="16475e0352609543452b124a4979663dd977f88b3bba7bebcaaa9378b7136ac7"
Sep 09 21:59:12.459274 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:59:12.459251    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16475e0352609543452b124a4979663dd977f88b3bba7bebcaaa9378b7136ac7\": container with ID starting with 16475e0352609543452b124a4979663dd977f88b3bba7bebcaaa9378b7136ac7 not found: ID does not exist" containerID="16475e0352609543452b124a4979663dd977f88b3bba7bebcaaa9378b7136ac7"
Sep 09 21:59:12.459370 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.459278    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16475e0352609543452b124a4979663dd977f88b3bba7bebcaaa9378b7136ac7"} err="failed to get container status \"16475e0352609543452b124a4979663dd977f88b3bba7bebcaaa9378b7136ac7\": rpc error: code = NotFound desc = could not find container \"16475e0352609543452b124a4979663dd977f88b3bba7bebcaaa9378b7136ac7\": container with ID starting with 16475e0352609543452b124a4979663dd977f88b3bba7bebcaaa9378b7136ac7 not found: ID does not exist"
Sep 09 21:59:12.459370 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.459296    2578 scope.go:117] "RemoveContainer" containerID="af1f598aba0235e89dd793bb0ff0bca71e507ca350c7804a4a7d464bc1a2f955"
Sep 09 21:59:12.459576 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:59:12.459555    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af1f598aba0235e89dd793bb0ff0bca71e507ca350c7804a4a7d464bc1a2f955\": container with ID starting with af1f598aba0235e89dd793bb0ff0bca71e507ca350c7804a4a7d464bc1a2f955 not found: ID does not exist" containerID="af1f598aba0235e89dd793bb0ff0bca71e507ca350c7804a4a7d464bc1a2f955"
Sep 09 21:59:12.459638 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.459581    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af1f598aba0235e89dd793bb0ff0bca71e507ca350c7804a4a7d464bc1a2f955"} err="failed to get container status \"af1f598aba0235e89dd793bb0ff0bca71e507ca350c7804a4a7d464bc1a2f955\": rpc error: code = NotFound desc = could not find container \"af1f598aba0235e89dd793bb0ff0bca71e507ca350c7804a4a7d464bc1a2f955\": container with ID starting with af1f598aba0235e89dd793bb0ff0bca71e507ca350c7804a4a7d464bc1a2f955 not found: ID does not exist"
Sep 09 21:59:12.465713 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.465691    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"]
Sep 09 21:59:12.471329 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:12.471310    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-390abc-openshift-default-8647c4bc56-tjhbb"]
Sep 09 21:59:13.003624 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:13.003585    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06282b23-1694-4fc2-b3c3-c17e924cca00" path="/var/lib/kubelet/pods/06282b23-1694-4fc2-b3c3-c17e924cca00/volumes"
Sep 09 21:59:15.079175 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.079119    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:59:15.213059 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.213029    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-envoy\") pod \"4c7834d5-06cf-4316-b19a-4613bb375eb1\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") "
Sep 09 21:59:15.213236 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.213068    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c7834d5-06cf-4316-b19a-4613bb375eb1-istiod-ca-cert\") pod \"4c7834d5-06cf-4316-b19a-4613bb375eb1\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") "
Sep 09 21:59:15.213236 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.213086    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-credential-socket\") pod \"4c7834d5-06cf-4316-b19a-4613bb375eb1\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") "
Sep 09 21:59:15.213236 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.213114    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-data\") pod \"4c7834d5-06cf-4316-b19a-4613bb375eb1\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") "
Sep 09 21:59:15.213236 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.213158    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-token\") pod \"4c7834d5-06cf-4316-b19a-4613bb375eb1\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") "
Sep 09 21:59:15.213236 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.213190    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z56lp\" (UniqueName: \"kubernetes.io/projected/4c7834d5-06cf-4316-b19a-4613bb375eb1-kube-api-access-z56lp\") pod \"4c7834d5-06cf-4316-b19a-4613bb375eb1\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") "
Sep 09 21:59:15.213510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.213276    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-workload-certs\") pod \"4c7834d5-06cf-4316-b19a-4613bb375eb1\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") "
Sep 09 21:59:15.213510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.213420    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4c7834d5-06cf-4316-b19a-4613bb375eb1" (UID: "4c7834d5-06cf-4316-b19a-4613bb375eb1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:15.213510 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.213491    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-data" (OuterVolumeSpecName: "istio-data") pod "4c7834d5-06cf-4316-b19a-4613bb375eb1" (UID: "4c7834d5-06cf-4316-b19a-4613bb375eb1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:15.213680 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.213531    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4c7834d5-06cf-4316-b19a-4613bb375eb1" (UID: "4c7834d5-06cf-4316-b19a-4613bb375eb1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:15.213680 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.213563    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c7834d5-06cf-4316-b19a-4613bb375eb1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4c7834d5-06cf-4316-b19a-4613bb375eb1" (UID: "4c7834d5-06cf-4316-b19a-4613bb375eb1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:59:15.213680 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.213568    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-workload-socket\") pod \"4c7834d5-06cf-4316-b19a-4613bb375eb1\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") "
Sep 09 21:59:15.213680 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.213647    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-podinfo\") pod \"4c7834d5-06cf-4316-b19a-4613bb375eb1\" (UID: \"4c7834d5-06cf-4316-b19a-4613bb375eb1\") "
Sep 09 21:59:15.213873 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.213699    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4c7834d5-06cf-4316-b19a-4613bb375eb1" (UID: "4c7834d5-06cf-4316-b19a-4613bb375eb1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:15.214099 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.214079    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:15.214202 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.214104    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:15.214202 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.214119    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c7834d5-06cf-4316-b19a-4613bb375eb1-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:15.214202 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.214132    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:15.214202 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.214145    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:15.215294 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.215269    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4c7834d5-06cf-4316-b19a-4613bb375eb1" (UID: "4c7834d5-06cf-4316-b19a-4613bb375eb1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:15.215403 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.215370    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-token" (OuterVolumeSpecName: "istio-token") pod "4c7834d5-06cf-4316-b19a-4613bb375eb1" (UID: "4c7834d5-06cf-4316-b19a-4613bb375eb1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:59:15.215576 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.215558    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4c7834d5-06cf-4316-b19a-4613bb375eb1" (UID: "4c7834d5-06cf-4316-b19a-4613bb375eb1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:59:15.215659 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.215645    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c7834d5-06cf-4316-b19a-4613bb375eb1-kube-api-access-z56lp" (OuterVolumeSpecName: "kube-api-access-z56lp") pod "4c7834d5-06cf-4316-b19a-4613bb375eb1" (UID: "4c7834d5-06cf-4316-b19a-4613bb375eb1"). InnerVolumeSpecName "kube-api-access-z56lp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:59:15.315239 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.315207    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:15.315239 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.315234    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z56lp\" (UniqueName: \"kubernetes.io/projected/4c7834d5-06cf-4316-b19a-4613bb375eb1-kube-api-access-z56lp\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:15.315454 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.315248    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:15.315454 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.315260    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c7834d5-06cf-4316-b19a-4613bb375eb1-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:15.461844 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.461774    2578 generic.go:356] "Generic (PLEG): container finished" podID="4c7834d5-06cf-4316-b19a-4613bb375eb1" containerID="2a004b6a401ddeff62900630a4dbc3afae1b0d37a944e0fd668b1bfeb2ce9110" exitCode=0
Sep 09 21:59:15.461844 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.461817    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"
Sep 09 21:59:15.462047 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.461862    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4" event={"ID":"4c7834d5-06cf-4316-b19a-4613bb375eb1","Type":"ContainerDied","Data":"2a004b6a401ddeff62900630a4dbc3afae1b0d37a944e0fd668b1bfeb2ce9110"}
Sep 09 21:59:15.462047 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.461922    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4" event={"ID":"4c7834d5-06cf-4316-b19a-4613bb375eb1","Type":"ContainerDied","Data":"45f73579a0f00b44346d07550a2c5096637c303bbb4f063dad04795f5e6c8624"}
Sep 09 21:59:15.462047 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.461944    2578 scope.go:117] "RemoveContainer" containerID="2a004b6a401ddeff62900630a4dbc3afae1b0d37a944e0fd668b1bfeb2ce9110"
Sep 09 21:59:15.471802 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.471783    2578 scope.go:117] "RemoveContainer" containerID="941efd20e2e1e30fdd10f3a32866c0453cfa30c6bda2fbb436670d993bf0cf64"
Sep 09 21:59:15.480467 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.480451    2578 scope.go:117] "RemoveContainer" containerID="2a004b6a401ddeff62900630a4dbc3afae1b0d37a944e0fd668b1bfeb2ce9110"
Sep 09 21:59:15.480737 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:59:15.480717    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a004b6a401ddeff62900630a4dbc3afae1b0d37a944e0fd668b1bfeb2ce9110\": container with ID starting with 2a004b6a401ddeff62900630a4dbc3afae1b0d37a944e0fd668b1bfeb2ce9110 not found: ID does not exist" containerID="2a004b6a401ddeff62900630a4dbc3afae1b0d37a944e0fd668b1bfeb2ce9110"
Sep 09 21:59:15.480804 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.480744    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a004b6a401ddeff62900630a4dbc3afae1b0d37a944e0fd668b1bfeb2ce9110"} err="failed to get container status \"2a004b6a401ddeff62900630a4dbc3afae1b0d37a944e0fd668b1bfeb2ce9110\": rpc error: code = NotFound desc = could not find container \"2a004b6a401ddeff62900630a4dbc3afae1b0d37a944e0fd668b1bfeb2ce9110\": container with ID starting with 2a004b6a401ddeff62900630a4dbc3afae1b0d37a944e0fd668b1bfeb2ce9110 not found: ID does not exist"
Sep 09 21:59:15.480804 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.480761    2578 scope.go:117] "RemoveContainer" containerID="941efd20e2e1e30fdd10f3a32866c0453cfa30c6bda2fbb436670d993bf0cf64"
Sep 09 21:59:15.480998 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:59:15.480962    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"941efd20e2e1e30fdd10f3a32866c0453cfa30c6bda2fbb436670d993bf0cf64\": container with ID starting with 941efd20e2e1e30fdd10f3a32866c0453cfa30c6bda2fbb436670d993bf0cf64 not found: ID does not exist" containerID="941efd20e2e1e30fdd10f3a32866c0453cfa30c6bda2fbb436670d993bf0cf64"
Sep 09 21:59:15.481108 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.481006    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"941efd20e2e1e30fdd10f3a32866c0453cfa30c6bda2fbb436670d993bf0cf64"} err="failed to get container status \"941efd20e2e1e30fdd10f3a32866c0453cfa30c6bda2fbb436670d993bf0cf64\": rpc error: code = NotFound desc = could not find container \"941efd20e2e1e30fdd10f3a32866c0453cfa30c6bda2fbb436670d993bf0cf64\": container with ID starting with 941efd20e2e1e30fdd10f3a32866c0453cfa30c6bda2fbb436670d993bf0cf64 not found: ID does not exist"
Sep 09 21:59:15.492571 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.492548    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"]
Sep 09 21:59:15.502243 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:15.502221    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-5d8401-openshift-default-6b7665bfd8-wbbs4"]
Sep 09 21:59:17.003405 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:17.003374    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c7834d5-06cf-4316-b19a-4613bb375eb1" path="/var/lib/kubelet/pods/4c7834d5-06cf-4316-b19a-4613bb375eb1/volumes"
Sep 09 21:59:21.415140 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:21.415103    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:59:25.747550 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:25.747519    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"]
Sep 09 21:59:25.747926 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:25.747731    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j" podUID="7a540ba2-b7b7-466a-aa3b-16170599fdbd" containerName="maas-api" containerID="cri-o://8e409b058cdd38c73d7870ea9dfd90d2f1634312787515bd1b237b9beb43c39a" gracePeriod=30
Sep 09 21:59:26.005608 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.005550    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:59:26.130442 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.130408    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a540ba2-b7b7-466a-aa3b-16170599fdbd-maas-api-tls\") pod \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") "
Sep 09 21:59:26.130678 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.130488    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t9b2\" (UniqueName: \"kubernetes.io/projected/7a540ba2-b7b7-466a-aa3b-16170599fdbd-kube-api-access-6t9b2\") pod \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") "
Sep 09 21:59:26.130678 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.130580    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a540ba2-b7b7-466a-aa3b-16170599fdbd-openshift-service-ca\") pod \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") "
Sep 09 21:59:26.130820 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.130725    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a540ba2-b7b7-466a-aa3b-16170599fdbd-metrics-certs\") pod \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\" (UID: \"7a540ba2-b7b7-466a-aa3b-16170599fdbd\") "
Sep 09 21:59:26.130961 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.130940    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a540ba2-b7b7-466a-aa3b-16170599fdbd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7a540ba2-b7b7-466a-aa3b-16170599fdbd" (UID: "7a540ba2-b7b7-466a-aa3b-16170599fdbd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:59:26.131149 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.131128    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a540ba2-b7b7-466a-aa3b-16170599fdbd-openshift-service-ca\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:26.133195 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.133153    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a540ba2-b7b7-466a-aa3b-16170599fdbd-kube-api-access-6t9b2" (OuterVolumeSpecName: "kube-api-access-6t9b2") pod "7a540ba2-b7b7-466a-aa3b-16170599fdbd" (UID: "7a540ba2-b7b7-466a-aa3b-16170599fdbd"). InnerVolumeSpecName "kube-api-access-6t9b2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:59:26.133443 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.133412    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a540ba2-b7b7-466a-aa3b-16170599fdbd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7a540ba2-b7b7-466a-aa3b-16170599fdbd" (UID: "7a540ba2-b7b7-466a-aa3b-16170599fdbd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:59:26.133554 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.133444    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a540ba2-b7b7-466a-aa3b-16170599fdbd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7a540ba2-b7b7-466a-aa3b-16170599fdbd" (UID: "7a540ba2-b7b7-466a-aa3b-16170599fdbd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:59:26.232457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.232370    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a540ba2-b7b7-466a-aa3b-16170599fdbd-metrics-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:26.232457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.232412    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a540ba2-b7b7-466a-aa3b-16170599fdbd-maas-api-tls\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:26.232457 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.232427    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t9b2\" (UniqueName: \"kubernetes.io/projected/7a540ba2-b7b7-466a-aa3b-16170599fdbd-kube-api-access-6t9b2\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:26.530183 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.530151    2578 generic.go:356] "Generic (PLEG): container finished" podID="7a540ba2-b7b7-466a-aa3b-16170599fdbd" containerID="8e409b058cdd38c73d7870ea9dfd90d2f1634312787515bd1b237b9beb43c39a" exitCode=0
Sep 09 21:59:26.530359 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.530194    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"
Sep 09 21:59:26.530359 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.530235    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j" event={"ID":"7a540ba2-b7b7-466a-aa3b-16170599fdbd","Type":"ContainerDied","Data":"8e409b058cdd38c73d7870ea9dfd90d2f1634312787515bd1b237b9beb43c39a"}
Sep 09 21:59:26.530359 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.530291    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j" event={"ID":"7a540ba2-b7b7-466a-aa3b-16170599fdbd","Type":"ContainerDied","Data":"a8ca5612d0b1d9e23f1e2667945581903e4607765d841cf648f2ec5eda80d7dd"}
Sep 09 21:59:26.530359 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.530313    2578 scope.go:117] "RemoveContainer" containerID="8e409b058cdd38c73d7870ea9dfd90d2f1634312787515bd1b237b9beb43c39a"
Sep 09 21:59:26.540422 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.540406    2578 scope.go:117] "RemoveContainer" containerID="8e409b058cdd38c73d7870ea9dfd90d2f1634312787515bd1b237b9beb43c39a"
Sep 09 21:59:26.540693 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:59:26.540672    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e409b058cdd38c73d7870ea9dfd90d2f1634312787515bd1b237b9beb43c39a\": container with ID starting with 8e409b058cdd38c73d7870ea9dfd90d2f1634312787515bd1b237b9beb43c39a not found: ID does not exist" containerID="8e409b058cdd38c73d7870ea9dfd90d2f1634312787515bd1b237b9beb43c39a"
Sep 09 21:59:26.540757 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.540701    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e409b058cdd38c73d7870ea9dfd90d2f1634312787515bd1b237b9beb43c39a"} err="failed to get container status \"8e409b058cdd38c73d7870ea9dfd90d2f1634312787515bd1b237b9beb43c39a\": rpc error: code = NotFound desc = could not find container \"8e409b058cdd38c73d7870ea9dfd90d2f1634312787515bd1b237b9beb43c39a\": container with ID starting with 8e409b058cdd38c73d7870ea9dfd90d2f1634312787515bd1b237b9beb43c39a not found: ID does not exist"
Sep 09 21:59:26.569012 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.568986    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"]
Sep 09 21:59:26.580182 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:26.580152    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9dedad-5f4cfbb779-mzn4j"]
Sep 09 21:59:27.004092 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:27.004015    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a540ba2-b7b7-466a-aa3b-16170599fdbd" path="/var/lib/kubelet/pods/7a540ba2-b7b7-466a-aa3b-16170599fdbd/volumes"
Sep 09 21:59:34.024853 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:34.024822    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"]
Sep 09 21:59:34.025267 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:34.025057    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx" podUID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" containerName="istio-proxy" containerID="cri-o://1b394c0404c3bf5c00835aed7cf3a7d086f0413b150712d7edd107471b4c250b" gracePeriod=30
Sep 09 21:59:36.777928 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:36.777894    2578 patch_prober.go:28] interesting pod/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 21:59:36.778334 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:36.777949    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx" podUID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 21:59:39.287086 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.287030    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:59:39.355889 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.355861    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-podinfo\") pod \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") "
Sep 09 21:59:39.356068 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.355894    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istiod-ca-cert\") pod \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") "
Sep 09 21:59:39.356068 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.355954    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-workload-certs\") pod \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") "
Sep 09 21:59:39.356068 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.355987    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-envoy\") pod \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") "
Sep 09 21:59:39.356068 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.356021    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-token\") pod \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") "
Sep 09 21:59:39.356068 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.356055    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-data\") pod \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") "
Sep 09 21:59:39.356388 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.356109    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-credential-socket\") pod \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") "
Sep 09 21:59:39.356388 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.356155    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-workload-socket\") pod \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") "
Sep 09 21:59:39.356388 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.356199    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xp8xx\" (UniqueName: \"kubernetes.io/projected/0e0b223d-abfc-42bd-ae44-1c828f2c2892-kube-api-access-xp8xx\") pod \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\" (UID: \"0e0b223d-abfc-42bd-ae44-1c828f2c2892\") "
Sep 09 21:59:39.356388 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.356289    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0e0b223d-abfc-42bd-ae44-1c828f2c2892" (UID: "0e0b223d-abfc-42bd-ae44-1c828f2c2892"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:39.356388 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.356337    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0e0b223d-abfc-42bd-ae44-1c828f2c2892" (UID: "0e0b223d-abfc-42bd-ae44-1c828f2c2892"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:59:39.356644 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.356552    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0e0b223d-abfc-42bd-ae44-1c828f2c2892" (UID: "0e0b223d-abfc-42bd-ae44-1c828f2c2892"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:39.356701 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.356607    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0e0b223d-abfc-42bd-ae44-1c828f2c2892" (UID: "0e0b223d-abfc-42bd-ae44-1c828f2c2892"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:39.356701 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.356620    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-data" (OuterVolumeSpecName: "istio-data") pod "0e0b223d-abfc-42bd-ae44-1c828f2c2892" (UID: "0e0b223d-abfc-42bd-ae44-1c828f2c2892"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:39.356701 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.356611    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:39.356701 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.356686    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:39.358225 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.358177    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0e0b223d-abfc-42bd-ae44-1c828f2c2892" (UID: "0e0b223d-abfc-42bd-ae44-1c828f2c2892"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:59:39.358353 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.358286    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-token" (OuterVolumeSpecName: "istio-token") pod "0e0b223d-abfc-42bd-ae44-1c828f2c2892" (UID: "0e0b223d-abfc-42bd-ae44-1c828f2c2892"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:59:39.358549 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.358519    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0e0b223d-abfc-42bd-ae44-1c828f2c2892" (UID: "0e0b223d-abfc-42bd-ae44-1c828f2c2892"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:39.358654 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.358635    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e0b223d-abfc-42bd-ae44-1c828f2c2892-kube-api-access-xp8xx" (OuterVolumeSpecName: "kube-api-access-xp8xx") pod "0e0b223d-abfc-42bd-ae44-1c828f2c2892" (UID: "0e0b223d-abfc-42bd-ae44-1c828f2c2892"). InnerVolumeSpecName "kube-api-access-xp8xx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:59:39.457336 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.457306    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:39.457336 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.457330    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:39.457531 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.457343    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:39.457531 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.457355    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:39.457531 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.457363    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e0b223d-abfc-42bd-ae44-1c828f2c2892-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:39.457531 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.457372    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xp8xx\" (UniqueName: \"kubernetes.io/projected/0e0b223d-abfc-42bd-ae44-1c828f2c2892-kube-api-access-xp8xx\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:39.457531 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.457381    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e0b223d-abfc-42bd-ae44-1c828f2c2892-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:39.615230 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.615151    2578 generic.go:356] "Generic (PLEG): container finished" podID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" containerID="1b394c0404c3bf5c00835aed7cf3a7d086f0413b150712d7edd107471b4c250b" exitCode=0
Sep 09 21:59:39.615230 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.615196    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"
Sep 09 21:59:39.615406 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.615228    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx" event={"ID":"0e0b223d-abfc-42bd-ae44-1c828f2c2892","Type":"ContainerDied","Data":"1b394c0404c3bf5c00835aed7cf3a7d086f0413b150712d7edd107471b4c250b"}
Sep 09 21:59:39.615406 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.615282    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx" event={"ID":"0e0b223d-abfc-42bd-ae44-1c828f2c2892","Type":"ContainerDied","Data":"8604296b7e08b9ee9bbfb8715542fecbb9470bbbe703744fdd23cfbfd58a7225"}
Sep 09 21:59:39.615406 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.615312    2578 scope.go:117] "RemoveContainer" containerID="1b394c0404c3bf5c00835aed7cf3a7d086f0413b150712d7edd107471b4c250b"
Sep 09 21:59:39.626146 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.626026    2578 scope.go:117] "RemoveContainer" containerID="0f2ecdd3dbcc1d1fcc2ae06e1445c49b2499cee8ab008f90b2a8f1b4f4595c9d"
Sep 09 21:59:39.635431 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.635404    2578 scope.go:117] "RemoveContainer" containerID="1b394c0404c3bf5c00835aed7cf3a7d086f0413b150712d7edd107471b4c250b"
Sep 09 21:59:39.635739 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:59:39.635717    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b394c0404c3bf5c00835aed7cf3a7d086f0413b150712d7edd107471b4c250b\": container with ID starting with 1b394c0404c3bf5c00835aed7cf3a7d086f0413b150712d7edd107471b4c250b not found: ID does not exist" containerID="1b394c0404c3bf5c00835aed7cf3a7d086f0413b150712d7edd107471b4c250b"
Sep 09 21:59:39.635824 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.635750    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b394c0404c3bf5c00835aed7cf3a7d086f0413b150712d7edd107471b4c250b"} err="failed to get container status \"1b394c0404c3bf5c00835aed7cf3a7d086f0413b150712d7edd107471b4c250b\": rpc error: code = NotFound desc = could not find container \"1b394c0404c3bf5c00835aed7cf3a7d086f0413b150712d7edd107471b4c250b\": container with ID starting with 1b394c0404c3bf5c00835aed7cf3a7d086f0413b150712d7edd107471b4c250b not found: ID does not exist"
Sep 09 21:59:39.635824 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.635772    2578 scope.go:117] "RemoveContainer" containerID="0f2ecdd3dbcc1d1fcc2ae06e1445c49b2499cee8ab008f90b2a8f1b4f4595c9d"
Sep 09 21:59:39.636114 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:59:39.636093    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f2ecdd3dbcc1d1fcc2ae06e1445c49b2499cee8ab008f90b2a8f1b4f4595c9d\": container with ID starting with 0f2ecdd3dbcc1d1fcc2ae06e1445c49b2499cee8ab008f90b2a8f1b4f4595c9d not found: ID does not exist" containerID="0f2ecdd3dbcc1d1fcc2ae06e1445c49b2499cee8ab008f90b2a8f1b4f4595c9d"
Sep 09 21:59:39.636191 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.636118    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f2ecdd3dbcc1d1fcc2ae06e1445c49b2499cee8ab008f90b2a8f1b4f4595c9d"} err="failed to get container status \"0f2ecdd3dbcc1d1fcc2ae06e1445c49b2499cee8ab008f90b2a8f1b4f4595c9d\": rpc error: code = NotFound desc = could not find container \"0f2ecdd3dbcc1d1fcc2ae06e1445c49b2499cee8ab008f90b2a8f1b4f4595c9d\": container with ID starting with 0f2ecdd3dbcc1d1fcc2ae06e1445c49b2499cee8ab008f90b2a8f1b4f4595c9d not found: ID does not exist"
Sep 09 21:59:39.640589 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.640566    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"]
Sep 09 21:59:39.645210 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:39.645189    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-9dedad-openshift-default-ccb56564-mppvx"]
Sep 09 21:59:41.003913 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:41.003878    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" path="/var/lib/kubelet/pods/0e0b223d-abfc-42bd-ae44-1c828f2c2892/volumes"
Sep 09 21:59:44.839703 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:44.839660    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"]
Sep 09 21:59:44.840241 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:44.839953    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x" podUID="6ee6a3c9-5b6a-4647-9771-0691097c2e35" containerName="maas-api" containerID="cri-o://df05e6e3c25db0ff04d6f725f737873bc7aacc9c68b6c471fe6576c5f7c311c1" gracePeriod=30
Sep 09 21:59:45.124683 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.124648    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:59:45.210406 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.210373    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee6a3c9-5b6a-4647-9771-0691097c2e35-openshift-service-ca\") pod \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") "
Sep 09 21:59:45.210406 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.210414    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ee6a3c9-5b6a-4647-9771-0691097c2e35-metrics-certs\") pod \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") "
Sep 09 21:59:45.210669 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.210463    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee6a3c9-5b6a-4647-9771-0691097c2e35-maas-api-tls\") pod \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") "
Sep 09 21:59:45.210669 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.210484    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbhgz\" (UniqueName: \"kubernetes.io/projected/6ee6a3c9-5b6a-4647-9771-0691097c2e35-kube-api-access-bbhgz\") pod \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\" (UID: \"6ee6a3c9-5b6a-4647-9771-0691097c2e35\") "
Sep 09 21:59:45.210917 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.210883    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ee6a3c9-5b6a-4647-9771-0691097c2e35-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6ee6a3c9-5b6a-4647-9771-0691097c2e35" (UID: "6ee6a3c9-5b6a-4647-9771-0691097c2e35"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:59:45.212516 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.212489    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ee6a3c9-5b6a-4647-9771-0691097c2e35-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6ee6a3c9-5b6a-4647-9771-0691097c2e35" (UID: "6ee6a3c9-5b6a-4647-9771-0691097c2e35"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:59:45.212603 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.212584    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ee6a3c9-5b6a-4647-9771-0691097c2e35-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6ee6a3c9-5b6a-4647-9771-0691097c2e35" (UID: "6ee6a3c9-5b6a-4647-9771-0691097c2e35"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:59:45.212685 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.212663    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ee6a3c9-5b6a-4647-9771-0691097c2e35-kube-api-access-bbhgz" (OuterVolumeSpecName: "kube-api-access-bbhgz") pod "6ee6a3c9-5b6a-4647-9771-0691097c2e35" (UID: "6ee6a3c9-5b6a-4647-9771-0691097c2e35"). InnerVolumeSpecName "kube-api-access-bbhgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:59:45.311740 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.311704    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ee6a3c9-5b6a-4647-9771-0691097c2e35-openshift-service-ca\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:45.311740 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.311737    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ee6a3c9-5b6a-4647-9771-0691097c2e35-metrics-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:45.311740 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.311748    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ee6a3c9-5b6a-4647-9771-0691097c2e35-maas-api-tls\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:45.312046 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.311756    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbhgz\" (UniqueName: \"kubernetes.io/projected/6ee6a3c9-5b6a-4647-9771-0691097c2e35-kube-api-access-bbhgz\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:45.656052 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.656016    2578 generic.go:356] "Generic (PLEG): container finished" podID="6ee6a3c9-5b6a-4647-9771-0691097c2e35" containerID="df05e6e3c25db0ff04d6f725f737873bc7aacc9c68b6c471fe6576c5f7c311c1" exitCode=0
Sep 09 21:59:45.656254 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.656070    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"
Sep 09 21:59:45.656254 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.656105    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x" event={"ID":"6ee6a3c9-5b6a-4647-9771-0691097c2e35","Type":"ContainerDied","Data":"df05e6e3c25db0ff04d6f725f737873bc7aacc9c68b6c471fe6576c5f7c311c1"}
Sep 09 21:59:45.656254 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.656163    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x" event={"ID":"6ee6a3c9-5b6a-4647-9771-0691097c2e35","Type":"ContainerDied","Data":"38076f2d6b619a10a2ee5be6b4663c236aa54c3682f54d39c96fb62f2bcf58da"}
Sep 09 21:59:45.656254 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.656187    2578 scope.go:117] "RemoveContainer" containerID="df05e6e3c25db0ff04d6f725f737873bc7aacc9c68b6c471fe6576c5f7c311c1"
Sep 09 21:59:45.669703 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.669683    2578 scope.go:117] "RemoveContainer" containerID="df05e6e3c25db0ff04d6f725f737873bc7aacc9c68b6c471fe6576c5f7c311c1"
Sep 09 21:59:45.670030 ip-10-0-140-157 kubenswrapper[2578]: E0909 21:59:45.670003    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df05e6e3c25db0ff04d6f725f737873bc7aacc9c68b6c471fe6576c5f7c311c1\": container with ID starting with df05e6e3c25db0ff04d6f725f737873bc7aacc9c68b6c471fe6576c5f7c311c1 not found: ID does not exist" containerID="df05e6e3c25db0ff04d6f725f737873bc7aacc9c68b6c471fe6576c5f7c311c1"
Sep 09 21:59:45.670101 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.670038    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df05e6e3c25db0ff04d6f725f737873bc7aacc9c68b6c471fe6576c5f7c311c1"} err="failed to get container status \"df05e6e3c25db0ff04d6f725f737873bc7aacc9c68b6c471fe6576c5f7c311c1\": rpc error: code = NotFound desc = could not find container \"df05e6e3c25db0ff04d6f725f737873bc7aacc9c68b6c471fe6576c5f7c311c1\": container with ID starting with df05e6e3c25db0ff04d6f725f737873bc7aacc9c68b6c471fe6576c5f7c311c1 not found: ID does not exist"
Sep 09 21:59:45.681138 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.681106    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"]
Sep 09 21:59:45.689373 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:45.689342    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-d99c83-7c778c9594-lln8x"]
Sep 09 21:59:47.004495 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:47.004455    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ee6a3c9-5b6a-4647-9771-0691097c2e35" path="/var/lib/kubelet/pods/6ee6a3c9-5b6a-4647-9771-0691097c2e35/volumes"
Sep 09 21:59:52.504048 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:52.504013    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"]
Sep 09 21:59:52.504494 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:52.504244    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg" podUID="23d09d41-47e5-4384-b6d1-df3b4087e544" containerName="istio-proxy" containerID="cri-o://5ab708ef4db0a607e237a4e67ecd7d1bbe87fa6372824bc0a44859948e1b4acb" gracePeriod=30
Sep 09 21:59:53.877879 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:53.877784    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 21:59:57.730999 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.730925    2578 generic.go:356] "Generic (PLEG): container finished" podID="23d09d41-47e5-4384-b6d1-df3b4087e544" containerID="5ab708ef4db0a607e237a4e67ecd7d1bbe87fa6372824bc0a44859948e1b4acb" exitCode=0
Sep 09 21:59:57.731383 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.731000    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg" event={"ID":"23d09d41-47e5-4384-b6d1-df3b4087e544","Type":"ContainerDied","Data":"5ab708ef4db0a607e237a4e67ecd7d1bbe87fa6372824bc0a44859948e1b4acb"}
Sep 09 21:59:57.731383 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.731076    2578 scope.go:117] "RemoveContainer" containerID="e28302ee4f3bc3ecb8909d0661414bc049d3810ac05e0bc56e5bd32d7d0765ca"
Sep 09 21:59:57.767846 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.767823    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:59:57.833316 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.833284    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-podinfo\") pod \"23d09d41-47e5-4384-b6d1-df3b4087e544\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") "
Sep 09 21:59:57.833478 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.833343    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-data\") pod \"23d09d41-47e5-4384-b6d1-df3b4087e544\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") "
Sep 09 21:59:57.833478 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.833391    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23d09d41-47e5-4384-b6d1-df3b4087e544-istiod-ca-cert\") pod \"23d09d41-47e5-4384-b6d1-df3b4087e544\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") "
Sep 09 21:59:57.833478 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.833456    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5h6kk\" (UniqueName: \"kubernetes.io/projected/23d09d41-47e5-4384-b6d1-df3b4087e544-kube-api-access-5h6kk\") pod \"23d09d41-47e5-4384-b6d1-df3b4087e544\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") "
Sep 09 21:59:57.833676 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.833516    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-workload-socket\") pod \"23d09d41-47e5-4384-b6d1-df3b4087e544\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") "
Sep 09 21:59:57.833676 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.833540    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-credential-socket\") pod \"23d09d41-47e5-4384-b6d1-df3b4087e544\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") "
Sep 09 21:59:57.833676 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.833568    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-envoy\") pod \"23d09d41-47e5-4384-b6d1-df3b4087e544\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") "
Sep 09 21:59:57.833676 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.833624    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-data" (OuterVolumeSpecName: "istio-data") pod "23d09d41-47e5-4384-b6d1-df3b4087e544" (UID: "23d09d41-47e5-4384-b6d1-df3b4087e544"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:57.833889 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.833814    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23d09d41-47e5-4384-b6d1-df3b4087e544-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "23d09d41-47e5-4384-b6d1-df3b4087e544" (UID: "23d09d41-47e5-4384-b6d1-df3b4087e544"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:59:57.833889 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.833819    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "23d09d41-47e5-4384-b6d1-df3b4087e544" (UID: "23d09d41-47e5-4384-b6d1-df3b4087e544"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:57.833889 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.833842    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "23d09d41-47e5-4384-b6d1-df3b4087e544" (UID: "23d09d41-47e5-4384-b6d1-df3b4087e544"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:57.834125 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.834096    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "23d09d41-47e5-4384-b6d1-df3b4087e544" (UID: "23d09d41-47e5-4384-b6d1-df3b4087e544"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:57.834257 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.833599    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-workload-certs\") pod \"23d09d41-47e5-4384-b6d1-df3b4087e544\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") "
Sep 09 21:59:57.834328 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.834265    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-token\") pod \"23d09d41-47e5-4384-b6d1-df3b4087e544\" (UID: \"23d09d41-47e5-4384-b6d1-df3b4087e544\") "
Sep 09 21:59:57.834645 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.834625    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:57.834740 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.834649    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:57.834740 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.834664    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:57.834740 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.834677    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:57.834740 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.834688    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23d09d41-47e5-4384-b6d1-df3b4087e544-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:57.835722 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.835697    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "23d09d41-47e5-4384-b6d1-df3b4087e544" (UID: "23d09d41-47e5-4384-b6d1-df3b4087e544"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:59:57.836045 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.836020    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "23d09d41-47e5-4384-b6d1-df3b4087e544" (UID: "23d09d41-47e5-4384-b6d1-df3b4087e544"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:59:57.836045 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.836031    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23d09d41-47e5-4384-b6d1-df3b4087e544-kube-api-access-5h6kk" (OuterVolumeSpecName: "kube-api-access-5h6kk") pod "23d09d41-47e5-4384-b6d1-df3b4087e544" (UID: "23d09d41-47e5-4384-b6d1-df3b4087e544"). InnerVolumeSpecName "kube-api-access-5h6kk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:59:57.836212 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.836196    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-token" (OuterVolumeSpecName: "istio-token") pod "23d09d41-47e5-4384-b6d1-df3b4087e544" (UID: "23d09d41-47e5-4384-b6d1-df3b4087e544"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:59:57.935568 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.935542    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:57.935568 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.935565    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:57.935713 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.935576    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5h6kk\" (UniqueName: \"kubernetes.io/projected/23d09d41-47e5-4384-b6d1-df3b4087e544-kube-api-access-5h6kk\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:57.935713 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:57.935587    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23d09d41-47e5-4384-b6d1-df3b4087e544-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 21:59:58.738947 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:58.738918    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"
Sep 09 21:59:58.738947 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:58.738929    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg" event={"ID":"23d09d41-47e5-4384-b6d1-df3b4087e544","Type":"ContainerDied","Data":"1cce8cf2669c029d962afaa1e99c60d8a68ed6b29b79a6640cf7768cd30096ff"}
Sep 09 21:59:58.739463 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:58.739003    2578 scope.go:117] "RemoveContainer" containerID="5ab708ef4db0a607e237a4e67ecd7d1bbe87fa6372824bc0a44859948e1b4acb"
Sep 09 21:59:58.762367 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:58.762344    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"]
Sep 09 21:59:58.766897 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:58.766876    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-d99c83-openshift-default-77cb785cc4-8pzjg"]
Sep 09 21:59:59.004847 ip-10-0-140-157 kubenswrapper[2578]: I0909 21:59:59.004774    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23d09d41-47e5-4384-b6d1-df3b4087e544" path="/var/lib/kubelet/pods/23d09d41-47e5-4384-b6d1-df3b4087e544/volumes"
Sep 09 22:00:00.160419 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.160385    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816520-bxtb2"]
Sep 09 22:00:00.161022 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161005    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23d09d41-47e5-4384-b6d1-df3b4087e544" containerName="istio-proxy"
Sep 09 22:00:00.161022 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161023    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="23d09d41-47e5-4384-b6d1-df3b4087e544" containerName="istio-proxy"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161034    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23d09d41-47e5-4384-b6d1-df3b4087e544" containerName="istio-proxy"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161042    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="23d09d41-47e5-4384-b6d1-df3b4087e544" containerName="istio-proxy"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161051    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b" containerName="maas-api"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161059    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b" containerName="maas-api"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161069    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" containerName="istio-proxy"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161078    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" containerName="istio-proxy"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161089    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c7834d5-06cf-4316-b19a-4613bb375eb1" containerName="istio-proxy"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161097    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c7834d5-06cf-4316-b19a-4613bb375eb1" containerName="istio-proxy"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161113    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ee6a3c9-5b6a-4647-9771-0691097c2e35" containerName="maas-api"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161121    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ee6a3c9-5b6a-4647-9771-0691097c2e35" containerName="maas-api"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161141    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerName="istio-proxy"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161150    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerName="istio-proxy"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161168    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="110b9070-ec01-43e4-9985-626dbb6ce436" containerName="main"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161176    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="110b9070-ec01-43e4-9985-626dbb6ce436" containerName="main"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161191    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a540ba2-b7b7-466a-aa3b-16170599fdbd" containerName="maas-api"
Sep 09 22:00:00.161203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161199    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a540ba2-b7b7-466a-aa3b-16170599fdbd" containerName="maas-api"
Sep 09 22:00:00.161988 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161212    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c7834d5-06cf-4316-b19a-4613bb375eb1" containerName="istio-proxy"
Sep 09 22:00:00.161988 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161220    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c7834d5-06cf-4316-b19a-4613bb375eb1" containerName="istio-proxy"
Sep 09 22:00:00.161988 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161318    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a540ba2-b7b7-466a-aa3b-16170599fdbd" containerName="maas-api"
Sep 09 22:00:00.161988 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161329    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" containerName="istio-proxy"
Sep 09 22:00:00.161988 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161339    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="23d09d41-47e5-4384-b6d1-df3b4087e544" containerName="istio-proxy"
Sep 09 22:00:00.161988 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161348    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerName="istio-proxy"
Sep 09 22:00:00.161988 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161356    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c7834d5-06cf-4316-b19a-4613bb375eb1" containerName="istio-proxy"
Sep 09 22:00:00.161988 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161363    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ee6a3c9-5b6a-4647-9771-0691097c2e35" containerName="maas-api"
Sep 09 22:00:00.161988 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161374    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" containerName="istio-proxy"
Sep 09 22:00:00.161988 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161382    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="110b9070-ec01-43e4-9985-626dbb6ce436" containerName="main"
Sep 09 22:00:00.161988 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.161391    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="73ea1bbd-d8d2-4b8a-9dee-1fe10ea6e99b" containerName="maas-api"
Sep 09 22:00:00.166412 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.166389    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816520-bxtb2"
Sep 09 22:00:00.169819 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.169794    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65jqq\""
Sep 09 22:00:00.174309 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.173992    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2d3bc8-29816520-lmb48"]
Sep 09 22:00:00.174515 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.174494    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerName="istio-proxy"
Sep 09 22:00:00.174610 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.174518    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerName="istio-proxy"
Sep 09 22:00:00.174610 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.174561    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" containerName="istio-proxy"
Sep 09 22:00:00.174610 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.174571    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e0b223d-abfc-42bd-ae44-1c828f2c2892" containerName="istio-proxy"
Sep 09 22:00:00.174764 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.174697    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="23d09d41-47e5-4384-b6d1-df3b4087e544" containerName="istio-proxy"
Sep 09 22:00:00.174764 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.174709    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="06282b23-1694-4fc2-b3c3-c17e924cca00" containerName="istio-proxy"
Sep 09 22:00:00.174764 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.174718    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c7834d5-06cf-4316-b19a-4613bb375eb1" containerName="istio-proxy"
Sep 09 22:00:00.178401 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.178380    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816520-bxtb2"
Sep 09 22:00:00.179052 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.179032    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2d3bc8-29816520-lmb48"
Sep 09 22:00:00.179269 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.179142    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816520-bxtb2"]
Sep 09 22:00:00.192157 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.192117    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2d3bc8-29816520-lmb48"]
Sep 09 22:00:00.217012 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.216960    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2d3bc8-29816520-lmb48"
Sep 09 22:00:00.465865 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.465836    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816520-bxtb2"]
Sep 09 22:00:00.466936 ip-10-0-140-157 kubenswrapper[2578]: W0909 22:00:00.466910    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80370a40_f00b_45e0_846e_a80189ee0cc4.slice/crio-c714b12c8581ca48c32abd1cc0a71f3cdb982d9dcc0bc9d341108a44156a797e WatchSource:0}: Error finding container c714b12c8581ca48c32abd1cc0a71f3cdb982d9dcc0bc9d341108a44156a797e: Status 404 returned error can't find the container with id c714b12c8581ca48c32abd1cc0a71f3cdb982d9dcc0bc9d341108a44156a797e
Sep 09 22:00:00.510148 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.510125    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2d3bc8-29816520-lmb48"]
Sep 09 22:00:00.512030 ip-10-0-140-157 kubenswrapper[2578]: W0909 22:00:00.512006    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a1328b1_f64f_4f75_af79_0e6e07926705.slice/crio-4f9de064b8b30c8c2997b632e091c45db44caf1321ea259b6bf833acf39caa57 WatchSource:0}: Error finding container 4f9de064b8b30c8c2997b632e091c45db44caf1321ea259b6bf833acf39caa57: Status 404 returned error can't find the container with id 4f9de064b8b30c8c2997b632e091c45db44caf1321ea259b6bf833acf39caa57
Sep 09 22:00:00.754546 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.754521    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816520-bxtb2" event={"ID":"80370a40-f00b-45e0-846e-a80189ee0cc4","Type":"ContainerStarted","Data":"28b3a8e91fe7a37fc152e9c2f0813fedbf7cd27907c8d639879db9a91ed40fd5"}
Sep 09 22:00:00.754658 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.754569    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816520-bxtb2" event={"ID":"80370a40-f00b-45e0-846e-a80189ee0cc4","Type":"ContainerStarted","Data":"c714b12c8581ca48c32abd1cc0a71f3cdb982d9dcc0bc9d341108a44156a797e"}
Sep 09 22:00:00.756163 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.756084    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2d3bc8-29816520-lmb48" event={"ID":"8a1328b1-f64f-4f75-af79-0e6e07926705","Type":"ContainerStarted","Data":"4f9de064b8b30c8c2997b632e091c45db44caf1321ea259b6bf833acf39caa57"}
Sep 09 22:00:00.770841 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:00.770798    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816520-bxtb2" podStartSLOduration=0.770782978 podStartE2EDuration="770.782978ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:00:00.769296261 +0000 UTC m=+1620.242261171" watchObservedRunningTime="2026-09-09 22:00:00.770782978 +0000 UTC m=+1620.243747888"
Sep 09 22:00:01.763333 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:01.763241    2578 generic.go:356] "Generic (PLEG): container finished" podID="8a1328b1-f64f-4f75-af79-0e6e07926705" containerID="c3891adb328764ed04ddaf23a25f55918d925f615c0b94b1a529aeacbb83ed61" exitCode=0
Sep 09 22:00:01.763730 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:01.763330    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2d3bc8-29816520-lmb48" event={"ID":"8a1328b1-f64f-4f75-af79-0e6e07926705","Type":"ContainerDied","Data":"c3891adb328764ed04ddaf23a25f55918d925f615c0b94b1a529aeacbb83ed61"}
Sep 09 22:00:01.764901 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:01.764875    2578 generic.go:356] "Generic (PLEG): container finished" podID="80370a40-f00b-45e0-846e-a80189ee0cc4" containerID="28b3a8e91fe7a37fc152e9c2f0813fedbf7cd27907c8d639879db9a91ed40fd5" exitCode=0
Sep 09 22:00:01.765043 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:01.764951    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816520-bxtb2" event={"ID":"80370a40-f00b-45e0-846e-a80189ee0cc4","Type":"ContainerDied","Data":"28b3a8e91fe7a37fc152e9c2f0813fedbf7cd27907c8d639879db9a91ed40fd5"}
Sep 09 22:00:02.912181 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:02.912159    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816520-bxtb2"
Sep 09 22:00:02.939184 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:02.939163    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2d3bc8-29816520-lmb48"
Sep 09 22:00:03.777894 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:03.777870    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2d3bc8-29816520-lmb48"
Sep 09 22:00:03.778092 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:03.777905    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2d3bc8-29816520-lmb48" event={"ID":"8a1328b1-f64f-4f75-af79-0e6e07926705","Type":"ContainerDied","Data":"4f9de064b8b30c8c2997b632e091c45db44caf1321ea259b6bf833acf39caa57"}
Sep 09 22:00:03.778092 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:03.777956    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f9de064b8b30c8c2997b632e091c45db44caf1321ea259b6bf833acf39caa57"
Sep 09 22:00:03.779460 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:03.779439    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816520-bxtb2"
Sep 09 22:00:03.779599 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:03.779462    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816520-bxtb2" event={"ID":"80370a40-f00b-45e0-846e-a80189ee0cc4","Type":"ContainerDied","Data":"c714b12c8581ca48c32abd1cc0a71f3cdb982d9dcc0bc9d341108a44156a797e"}
Sep 09 22:00:03.779599 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:03.779499    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c714b12c8581ca48c32abd1cc0a71f3cdb982d9dcc0bc9d341108a44156a797e"
Sep 09 22:00:10.590581 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:10.590542    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"]
Sep 09 22:00:10.591052 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:10.590796    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4" podUID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" containerName="istio-proxy" containerID="cri-o://dd050cbb47a0a0cb32179d2c32aafd8ca6fceb5222fb17ba5e9486c8d18517ce" gracePeriod=30
Sep 09 22:00:12.770481 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:12.770442    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 22:00:12.770857 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:12.770502    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4" podUID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 22:00:15.851722 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:15.851637    2578 generic.go:356] "Generic (PLEG): container finished" podID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" containerID="dd050cbb47a0a0cb32179d2c32aafd8ca6fceb5222fb17ba5e9486c8d18517ce" exitCode=0
Sep 09 22:00:15.852139 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:15.851723    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4" event={"ID":"d783cd0c-d8a3-44c7-b9fd-39c0821873fa","Type":"ContainerDied","Data":"dd050cbb47a0a0cb32179d2c32aafd8ca6fceb5222fb17ba5e9486c8d18517ce"}
Sep 09 22:00:15.852139 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:15.851796    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4" event={"ID":"d783cd0c-d8a3-44c7-b9fd-39c0821873fa","Type":"ContainerDied","Data":"27f15a402fdefcfb945da265c1757eba576accbef8316a35cfea7b6e753c4014"}
Sep 09 22:00:15.852139 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:15.851824    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27f15a402fdefcfb945da265c1757eba576accbef8316a35cfea7b6e753c4014"
Sep 09 22:00:15.852139 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:15.851844    2578 scope.go:117] "RemoveContainer" containerID="8e44b2db8ab47322f61d60b8faff767f27de6882cbdfb88cde7851c1781c634e"
Sep 09 22:00:15.854662 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:15.854638    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 22:00:16.017397 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.017363    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-credential-socket\") pod \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") "
Sep 09 22:00:16.017592 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.017407    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istiod-ca-cert\") pod \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") "
Sep 09 22:00:16.017592 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.017448    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-workload-socket\") pod \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") "
Sep 09 22:00:16.017592 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.017478    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-workload-certs\") pod \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") "
Sep 09 22:00:16.017592 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.017516    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-envoy\") pod \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") "
Sep 09 22:00:16.017592 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.017552    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-token\") pod \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") "
Sep 09 22:00:16.017858 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.017593    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ph8x\" (UniqueName: \"kubernetes.io/projected/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-kube-api-access-9ph8x\") pod \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") "
Sep 09 22:00:16.017858 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.017619    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-data\") pod \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") "
Sep 09 22:00:16.017858 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.017706    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-podinfo\") pod \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\" (UID: \"d783cd0c-d8a3-44c7-b9fd-39c0821873fa\") "
Sep 09 22:00:16.017858 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.017755    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d783cd0c-d8a3-44c7-b9fd-39c0821873fa" (UID: "d783cd0c-d8a3-44c7-b9fd-39c0821873fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:00:16.017858 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.017763    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d783cd0c-d8a3-44c7-b9fd-39c0821873fa" (UID: "d783cd0c-d8a3-44c7-b9fd-39c0821873fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:00:16.017858 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.017834    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d783cd0c-d8a3-44c7-b9fd-39c0821873fa" (UID: "d783cd0c-d8a3-44c7-b9fd-39c0821873fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:00:16.018226 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.017958    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-data" (OuterVolumeSpecName: "istio-data") pod "d783cd0c-d8a3-44c7-b9fd-39c0821873fa" (UID: "d783cd0c-d8a3-44c7-b9fd-39c0821873fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:00:16.018226 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.017877    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d783cd0c-d8a3-44c7-b9fd-39c0821873fa" (UID: "d783cd0c-d8a3-44c7-b9fd-39c0821873fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 22:00:16.018226 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.018155    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:00:16.018226 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.018173    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:00:16.018226 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.018189    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:00:16.018226 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.018201    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:00:16.018226 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.018214    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:00:16.019844 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.019811    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d783cd0c-d8a3-44c7-b9fd-39c0821873fa" (UID: "d783cd0c-d8a3-44c7-b9fd-39c0821873fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:00:16.019956 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.019897    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-kube-api-access-9ph8x" (OuterVolumeSpecName: "kube-api-access-9ph8x") pod "d783cd0c-d8a3-44c7-b9fd-39c0821873fa" (UID: "d783cd0c-d8a3-44c7-b9fd-39c0821873fa"). InnerVolumeSpecName "kube-api-access-9ph8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:00:16.019956 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.019905    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-token" (OuterVolumeSpecName: "istio-token") pod "d783cd0c-d8a3-44c7-b9fd-39c0821873fa" (UID: "d783cd0c-d8a3-44c7-b9fd-39c0821873fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:00:16.020052 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.019961    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d783cd0c-d8a3-44c7-b9fd-39c0821873fa" (UID: "d783cd0c-d8a3-44c7-b9fd-39c0821873fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 22:00:16.119732 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.119636    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:00:16.119732 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.119671    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:00:16.119732 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.119680    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:00:16.119732 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.119689    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ph8x\" (UniqueName: \"kubernetes.io/projected/d783cd0c-d8a3-44c7-b9fd-39c0821873fa-kube-api-access-9ph8x\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:00:16.859933 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.859901    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"
Sep 09 22:00:16.896279 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.896244    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"]
Sep 09 22:00:16.900394 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:16.900371    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-9e275c-openshift-default-56594987bb-vfrx4"]
Sep 09 22:00:17.003135 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:17.003103    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" path="/var/lib/kubelet/pods/d783cd0c-d8a3-44c7-b9fd-39c0821873fa/volumes"
Sep 09 22:00:24.875344 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:24.875300    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:00:49.372399 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:00:49.372360    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:01:01.375791 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:01.375760    2578 scope.go:117] "RemoveContainer" containerID="dd050cbb47a0a0cb32179d2c32aafd8ca6fceb5222fb17ba5e9486c8d18517ce"
Sep 09 22:01:17.780212 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:17.780173    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:01:30.457258 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.457181    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7495-69f2dd"]
Sep 09 22:01:30.457620 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.457591    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a1328b1-f64f-4f75-af79-0e6e07926705" containerName="cleanup"
Sep 09 22:01:30.457620 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.457603    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a1328b1-f64f-4f75-af79-0e6e07926705" containerName="cleanup"
Sep 09 22:01:30.457620 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.457611    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" containerName="istio-proxy"
Sep 09 22:01:30.457620 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.457617    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" containerName="istio-proxy"
Sep 09 22:01:30.457755 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.457636    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" containerName="istio-proxy"
Sep 09 22:01:30.457755 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.457641    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" containerName="istio-proxy"
Sep 09 22:01:30.457755 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.457650    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80370a40-f00b-45e0-846e-a80189ee0cc4" containerName="cleanup"
Sep 09 22:01:30.457755 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.457655    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="80370a40-f00b-45e0-846e-a80189ee0cc4" containerName="cleanup"
Sep 09 22:01:30.457755 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.457720    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a1328b1-f64f-4f75-af79-0e6e07926705" containerName="cleanup"
Sep 09 22:01:30.457755 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.457729    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" containerName="istio-proxy"
Sep 09 22:01:30.457755 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.457736    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="80370a40-f00b-45e0-846e-a80189ee0cc4" containerName="cleanup"
Sep 09 22:01:30.460891 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.460875    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7495-69f2dd"
Sep 09 22:01:30.463341 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.463317    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-k4l58\""
Sep 09 22:01:30.466454 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.466427    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7495-69f2dd"]
Sep 09 22:01:30.616712 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.616677    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xdh2\" (UniqueName: \"kubernetes.io/projected/c4c49e97-1095-4f2c-a253-89cf2917fd5a-kube-api-access-6xdh2\") pod \"test-curl-7495-69f2dd\" (UID: \"c4c49e97-1095-4f2c-a253-89cf2917fd5a\") " pod="opendatahub/test-curl-7495-69f2dd"
Sep 09 22:01:30.717879 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.717792    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xdh2\" (UniqueName: \"kubernetes.io/projected/c4c49e97-1095-4f2c-a253-89cf2917fd5a-kube-api-access-6xdh2\") pod \"test-curl-7495-69f2dd\" (UID: \"c4c49e97-1095-4f2c-a253-89cf2917fd5a\") " pod="opendatahub/test-curl-7495-69f2dd"
Sep 09 22:01:30.725570 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.725543    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xdh2\" (UniqueName: \"kubernetes.io/projected/c4c49e97-1095-4f2c-a253-89cf2917fd5a-kube-api-access-6xdh2\") pod \"test-curl-7495-69f2dd\" (UID: \"c4c49e97-1095-4f2c-a253-89cf2917fd5a\") " pod="opendatahub/test-curl-7495-69f2dd"
Sep 09 22:01:30.771703 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:30.771676    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7495-69f2dd"
Sep 09 22:01:31.041321 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:31.041298    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7495-69f2dd"]
Sep 09 22:01:31.043066 ip-10-0-140-157 kubenswrapper[2578]: W0909 22:01:31.043038    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4c49e97_1095_4f2c_a253_89cf2917fd5a.slice/crio-d811f78990d7c4cff0351d85c39266ea362edbfafa4e78b6b6a44bc3f05177e6 WatchSource:0}: Error finding container d811f78990d7c4cff0351d85c39266ea362edbfafa4e78b6b6a44bc3f05177e6: Status 404 returned error can't find the container with id d811f78990d7c4cff0351d85c39266ea362edbfafa4e78b6b6a44bc3f05177e6
Sep 09 22:01:31.300842 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:31.300771    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7495-69f2dd" event={"ID":"c4c49e97-1095-4f2c-a253-89cf2917fd5a","Type":"ContainerStarted","Data":"9084f91b25a1220d947d9a7de2c03767074ead5a2617cd62a9222df0bed1f349"}
Sep 09 22:01:31.300842 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:31.300808    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7495-69f2dd" event={"ID":"c4c49e97-1095-4f2c-a253-89cf2917fd5a","Type":"ContainerStarted","Data":"d811f78990d7c4cff0351d85c39266ea362edbfafa4e78b6b6a44bc3f05177e6"}
Sep 09 22:01:31.330182 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:31.330136    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7495-69f2dd" podStartSLOduration=1.330122206 podStartE2EDuration="1.330122206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:01:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:01:31.327153938 +0000 UTC m=+1710.800118848" watchObservedRunningTime="2026-09-09 22:01:31.330122206 +0000 UTC m=+1710.803087170"
Sep 09 22:01:31.907981 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:31.907946    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7495-69f2dd"]
Sep 09 22:01:31.910728 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:31.910696    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7495-69f2dd"]
Sep 09 22:01:32.306240 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:01:32.306210    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7495-69f2dd" container="test-curl-7495-69f2dd"
Sep 09 22:01:33.313850 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:33.313803    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7495-69f2dd" podUID="c4c49e97-1095-4f2c-a253-89cf2917fd5a" containerName="test-curl-7495-69f2dd" containerID="cri-o://9084f91b25a1220d947d9a7de2c03767074ead5a2617cd62a9222df0bed1f349" gracePeriod=2
Sep 09 22:01:35.480138 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:35.480117    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7495-69f2dd"
Sep 09 22:01:35.559907 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:35.559882    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xdh2\" (UniqueName: \"kubernetes.io/projected/c4c49e97-1095-4f2c-a253-89cf2917fd5a-kube-api-access-6xdh2\") pod \"c4c49e97-1095-4f2c-a253-89cf2917fd5a\" (UID: \"c4c49e97-1095-4f2c-a253-89cf2917fd5a\") "
Sep 09 22:01:35.561947 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:35.561926    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4c49e97-1095-4f2c-a253-89cf2917fd5a-kube-api-access-6xdh2" (OuterVolumeSpecName: "kube-api-access-6xdh2") pod "c4c49e97-1095-4f2c-a253-89cf2917fd5a" (UID: "c4c49e97-1095-4f2c-a253-89cf2917fd5a"). InnerVolumeSpecName "kube-api-access-6xdh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:01:35.661433 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:35.661405    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xdh2\" (UniqueName: \"kubernetes.io/projected/c4c49e97-1095-4f2c-a253-89cf2917fd5a-kube-api-access-6xdh2\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:01:36.335570 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:36.335539    2578 generic.go:356] "Generic (PLEG): container finished" podID="c4c49e97-1095-4f2c-a253-89cf2917fd5a" containerID="9084f91b25a1220d947d9a7de2c03767074ead5a2617cd62a9222df0bed1f349" exitCode=137
Sep 09 22:01:36.335732 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:36.335584    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7495-69f2dd"
Sep 09 22:01:36.335732 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:36.335648    2578 scope.go:117] "RemoveContainer" containerID="9084f91b25a1220d947d9a7de2c03767074ead5a2617cd62a9222df0bed1f349"
Sep 09 22:01:36.344907 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:36.344882    2578 scope.go:117] "RemoveContainer" containerID="9084f91b25a1220d947d9a7de2c03767074ead5a2617cd62a9222df0bed1f349"
Sep 09 22:01:36.345214 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:01:36.345193    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9084f91b25a1220d947d9a7de2c03767074ead5a2617cd62a9222df0bed1f349\": container with ID starting with 9084f91b25a1220d947d9a7de2c03767074ead5a2617cd62a9222df0bed1f349 not found: ID does not exist" containerID="9084f91b25a1220d947d9a7de2c03767074ead5a2617cd62a9222df0bed1f349"
Sep 09 22:01:36.345290 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:36.345226    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9084f91b25a1220d947d9a7de2c03767074ead5a2617cd62a9222df0bed1f349"} err="failed to get container status \"9084f91b25a1220d947d9a7de2c03767074ead5a2617cd62a9222df0bed1f349\": rpc error: code = NotFound desc = could not find container \"9084f91b25a1220d947d9a7de2c03767074ead5a2617cd62a9222df0bed1f349\": container with ID starting with 9084f91b25a1220d947d9a7de2c03767074ead5a2617cd62a9222df0bed1f349 not found: ID does not exist"
Sep 09 22:01:36.345596 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:01:36.345577    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7495-69f2dd\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="c4c49e97-1095-4f2c-a253-89cf2917fd5a" pod="opendatahub/test-curl-7495-69f2dd"
Sep 09 22:01:37.003416 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:37.003382    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4c49e97-1095-4f2c-a253-89cf2917fd5a" path="/var/lib/kubelet/pods/c4c49e97-1095-4f2c-a253-89cf2917fd5a/volumes"
Sep 09 22:01:41.168419 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:01:41.168380    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:02:03.367892 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:03.367859    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:02:14.749524 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:14.749482    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"]
Sep 09 22:02:14.765931 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:14.765895    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"]
Sep 09 22:02:21.079707 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:21.079674    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:02:29.770146 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:29.770103    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh" podUID="48a06290-8726-4a1f-985a-fbfa0923bdb5" containerName="main" containerID="cri-o://d4dc1bf76c78466aa9fa87e377f0e71dcdf9452bb58b5b8a984609ae9394b4af" gracePeriod=45
Sep 09 22:02:29.785245 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:29.785214    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c" podUID="fd921041-9f42-4bc7-945c-50cb1f428df5" containerName="main" containerID="cri-o://2bd9410a00b5d1ccf7e1c514297157f7718e55149defb76d39251329fab30e07" gracePeriod=45
Sep 09 22:02:30.056348 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.056326    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 22:02:30.059486 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.059462    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 22:02:30.145424 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.145384    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-model-cache\") pod \"fd921041-9f42-4bc7-945c-50cb1f428df5\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") "
Sep 09 22:02:30.145648 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.145477    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48a06290-8726-4a1f-985a-fbfa0923bdb5-tls-certs\") pod \"48a06290-8726-4a1f-985a-fbfa0923bdb5\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") "
Sep 09 22:02:30.145648 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.145502    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd921041-9f42-4bc7-945c-50cb1f428df5-tls-certs\") pod \"fd921041-9f42-4bc7-945c-50cb1f428df5\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") "
Sep 09 22:02:30.145648 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.145537    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-dshm\") pod \"48a06290-8726-4a1f-985a-fbfa0923bdb5\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") "
Sep 09 22:02:30.145648 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.145563    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-home\") pod \"fd921041-9f42-4bc7-945c-50cb1f428df5\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") "
Sep 09 22:02:30.145648 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.145604    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ds7kb\" (UniqueName: \"kubernetes.io/projected/fd921041-9f42-4bc7-945c-50cb1f428df5-kube-api-access-ds7kb\") pod \"fd921041-9f42-4bc7-945c-50cb1f428df5\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") "
Sep 09 22:02:30.145648 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.145646    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-tmp-dir\") pod \"48a06290-8726-4a1f-985a-fbfa0923bdb5\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") "
Sep 09 22:02:30.146080 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.145687    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-dshm\") pod \"fd921041-9f42-4bc7-945c-50cb1f428df5\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") "
Sep 09 22:02:30.146080 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.145683    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-model-cache" (OuterVolumeSpecName: "model-cache") pod "fd921041-9f42-4bc7-945c-50cb1f428df5" (UID: "fd921041-9f42-4bc7-945c-50cb1f428df5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:02:30.146080 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.145723    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-home\") pod \"48a06290-8726-4a1f-985a-fbfa0923bdb5\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") "
Sep 09 22:02:30.146080 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.145746    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-model-cache\") pod \"48a06290-8726-4a1f-985a-fbfa0923bdb5\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") "
Sep 09 22:02:30.146080 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.145801    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5tm8\" (UniqueName: \"kubernetes.io/projected/48a06290-8726-4a1f-985a-fbfa0923bdb5-kube-api-access-k5tm8\") pod \"48a06290-8726-4a1f-985a-fbfa0923bdb5\" (UID: \"48a06290-8726-4a1f-985a-fbfa0923bdb5\") "
Sep 09 22:02:30.146080 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.145827    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-tmp-dir\") pod \"fd921041-9f42-4bc7-945c-50cb1f428df5\" (UID: \"fd921041-9f42-4bc7-945c-50cb1f428df5\") "
Sep 09 22:02:30.146080 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.145911    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-home" (OuterVolumeSpecName: "home") pod "fd921041-9f42-4bc7-945c-50cb1f428df5" (UID: "fd921041-9f42-4bc7-945c-50cb1f428df5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:02:30.146456 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.146380    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-model-cache\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:30.146456 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.146402    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-home\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:30.147007 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.146845    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fd921041-9f42-4bc7-945c-50cb1f428df5" (UID: "fd921041-9f42-4bc7-945c-50cb1f428df5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:02:30.147165 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.147142    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "48a06290-8726-4a1f-985a-fbfa0923bdb5" (UID: "48a06290-8726-4a1f-985a-fbfa0923bdb5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:02:30.147353 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.147334    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-model-cache" (OuterVolumeSpecName: "model-cache") pod "48a06290-8726-4a1f-985a-fbfa0923bdb5" (UID: "48a06290-8726-4a1f-985a-fbfa0923bdb5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:02:30.147627 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.147585    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-home" (OuterVolumeSpecName: "home") pod "48a06290-8726-4a1f-985a-fbfa0923bdb5" (UID: "48a06290-8726-4a1f-985a-fbfa0923bdb5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:02:30.149701 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.149286    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48a06290-8726-4a1f-985a-fbfa0923bdb5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "48a06290-8726-4a1f-985a-fbfa0923bdb5" (UID: "48a06290-8726-4a1f-985a-fbfa0923bdb5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 22:02:30.149815 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.149694    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-dshm" (OuterVolumeSpecName: "dshm") pod "48a06290-8726-4a1f-985a-fbfa0923bdb5" (UID: "48a06290-8726-4a1f-985a-fbfa0923bdb5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:02:30.149815 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.149802    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48a06290-8726-4a1f-985a-fbfa0923bdb5-kube-api-access-k5tm8" (OuterVolumeSpecName: "kube-api-access-k5tm8") pod "48a06290-8726-4a1f-985a-fbfa0923bdb5" (UID: "48a06290-8726-4a1f-985a-fbfa0923bdb5"). InnerVolumeSpecName "kube-api-access-k5tm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:02:30.150414 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.150387    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd921041-9f42-4bc7-945c-50cb1f428df5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fd921041-9f42-4bc7-945c-50cb1f428df5" (UID: "fd921041-9f42-4bc7-945c-50cb1f428df5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 22:02:30.150720 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.150695    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd921041-9f42-4bc7-945c-50cb1f428df5-kube-api-access-ds7kb" (OuterVolumeSpecName: "kube-api-access-ds7kb") pod "fd921041-9f42-4bc7-945c-50cb1f428df5" (UID: "fd921041-9f42-4bc7-945c-50cb1f428df5"). InnerVolumeSpecName "kube-api-access-ds7kb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:02:30.150831 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.150777    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-dshm" (OuterVolumeSpecName: "dshm") pod "fd921041-9f42-4bc7-945c-50cb1f428df5" (UID: "fd921041-9f42-4bc7-945c-50cb1f428df5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:02:30.247882 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.247856    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-dshm\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:30.247882 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.247882    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-home\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:30.248079 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.247895    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-model-cache\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:30.248079 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.247910    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5tm8\" (UniqueName: \"kubernetes.io/projected/48a06290-8726-4a1f-985a-fbfa0923bdb5-kube-api-access-k5tm8\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:30.248079 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.247922    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd921041-9f42-4bc7-945c-50cb1f428df5-tmp-dir\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:30.248079 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.247935    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48a06290-8726-4a1f-985a-fbfa0923bdb5-tls-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:30.248079 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.247946    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd921041-9f42-4bc7-945c-50cb1f428df5-tls-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:30.248079 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.247956    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-dshm\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:30.248079 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.247985    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds7kb\" (UniqueName: \"kubernetes.io/projected/fd921041-9f42-4bc7-945c-50cb1f428df5-kube-api-access-ds7kb\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:30.248079 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.247997    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48a06290-8726-4a1f-985a-fbfa0923bdb5-tmp-dir\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:30.658955 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.658926    2578 generic.go:356] "Generic (PLEG): container finished" podID="fd921041-9f42-4bc7-945c-50cb1f428df5" containerID="2bd9410a00b5d1ccf7e1c514297157f7718e55149defb76d39251329fab30e07" exitCode=0
Sep 09 22:02:30.659130 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.658986    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"
Sep 09 22:02:30.659130 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.659013    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c" event={"ID":"fd921041-9f42-4bc7-945c-50cb1f428df5","Type":"ContainerDied","Data":"2bd9410a00b5d1ccf7e1c514297157f7718e55149defb76d39251329fab30e07"}
Sep 09 22:02:30.659130 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.659050    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c" event={"ID":"fd921041-9f42-4bc7-945c-50cb1f428df5","Type":"ContainerDied","Data":"4af26026cb08cd08a5a8e902d2bfb8928a73f2f1744b03d442f21bc3a864969b"}
Sep 09 22:02:30.659130 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.659066    2578 scope.go:117] "RemoveContainer" containerID="2bd9410a00b5d1ccf7e1c514297157f7718e55149defb76d39251329fab30e07"
Sep 09 22:02:30.660610 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.660586    2578 generic.go:356] "Generic (PLEG): container finished" podID="48a06290-8726-4a1f-985a-fbfa0923bdb5" containerID="d4dc1bf76c78466aa9fa87e377f0e71dcdf9452bb58b5b8a984609ae9394b4af" exitCode=0
Sep 09 22:02:30.660725 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.660631    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"
Sep 09 22:02:30.660725 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.660669    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh" event={"ID":"48a06290-8726-4a1f-985a-fbfa0923bdb5","Type":"ContainerDied","Data":"d4dc1bf76c78466aa9fa87e377f0e71dcdf9452bb58b5b8a984609ae9394b4af"}
Sep 09 22:02:30.660725 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.660705    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh" event={"ID":"48a06290-8726-4a1f-985a-fbfa0923bdb5","Type":"ContainerDied","Data":"b157c773039f0d6f0ac4a8971321168bd9c588ebfb1ca1a3a93ddecfc9de9d18"}
Sep 09 22:02:30.669362 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.669340    2578 scope.go:117] "RemoveContainer" containerID="2bd9410a00b5d1ccf7e1c514297157f7718e55149defb76d39251329fab30e07"
Sep 09 22:02:30.669656 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:02:30.669628    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bd9410a00b5d1ccf7e1c514297157f7718e55149defb76d39251329fab30e07\": container with ID starting with 2bd9410a00b5d1ccf7e1c514297157f7718e55149defb76d39251329fab30e07 not found: ID does not exist" containerID="2bd9410a00b5d1ccf7e1c514297157f7718e55149defb76d39251329fab30e07"
Sep 09 22:02:30.669747 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.669666    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bd9410a00b5d1ccf7e1c514297157f7718e55149defb76d39251329fab30e07"} err="failed to get container status \"2bd9410a00b5d1ccf7e1c514297157f7718e55149defb76d39251329fab30e07\": rpc error: code = NotFound desc = could not find container \"2bd9410a00b5d1ccf7e1c514297157f7718e55149defb76d39251329fab30e07\": container with ID starting with 2bd9410a00b5d1ccf7e1c514297157f7718e55149defb76d39251329fab30e07 not found: ID does not exist"
Sep 09 22:02:30.669747 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.669689    2578 scope.go:117] "RemoveContainer" containerID="d4dc1bf76c78466aa9fa87e377f0e71dcdf9452bb58b5b8a984609ae9394b4af"
Sep 09 22:02:30.677819 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.677800    2578 scope.go:117] "RemoveContainer" containerID="d4dc1bf76c78466aa9fa87e377f0e71dcdf9452bb58b5b8a984609ae9394b4af"
Sep 09 22:02:30.678102 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:02:30.678080    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4dc1bf76c78466aa9fa87e377f0e71dcdf9452bb58b5b8a984609ae9394b4af\": container with ID starting with d4dc1bf76c78466aa9fa87e377f0e71dcdf9452bb58b5b8a984609ae9394b4af not found: ID does not exist" containerID="d4dc1bf76c78466aa9fa87e377f0e71dcdf9452bb58b5b8a984609ae9394b4af"
Sep 09 22:02:30.678179 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.678107    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4dc1bf76c78466aa9fa87e377f0e71dcdf9452bb58b5b8a984609ae9394b4af"} err="failed to get container status \"d4dc1bf76c78466aa9fa87e377f0e71dcdf9452bb58b5b8a984609ae9394b4af\": rpc error: code = NotFound desc = could not find container \"d4dc1bf76c78466aa9fa87e377f0e71dcdf9452bb58b5b8a984609ae9394b4af\": container with ID starting with d4dc1bf76c78466aa9fa87e377f0e71dcdf9452bb58b5b8a984609ae9394b4af not found: ID does not exist"
Sep 09 22:02:30.683689 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.683669    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"]
Sep 09 22:02:30.687847 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.687828    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-3fa986/embedding-w4-3fa986-kserve-8459dc5dbc-5cb8c"]
Sep 09 22:02:30.698290 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.698271    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"]
Sep 09 22:02:30.702096 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:30.702077    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-3fa986/distinct-2-w4-3fa986-kserve-7c47fc8b45-2xggh"]
Sep 09 22:02:31.004298 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:31.004225    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48a06290-8726-4a1f-985a-fbfa0923bdb5" path="/var/lib/kubelet/pods/48a06290-8726-4a1f-985a-fbfa0923bdb5/volumes"
Sep 09 22:02:31.004643 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:31.004613    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd921041-9f42-4bc7-945c-50cb1f428df5" path="/var/lib/kubelet/pods/fd921041-9f42-4bc7-945c-50cb1f428df5/volumes"
Sep 09 22:02:32.463914 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:32.463882    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"]
Sep 09 22:02:32.464339 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:32.464118    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerName="istio-proxy" containerID="cri-o://4a2c4ca5e012bf4c0c3098adf82f0c01778f6df30a6a3c404b0d35ec796f6632" gracePeriod=30
Sep 09 22:02:35.141651 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:35.141607    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 22:02:35.142142 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:35.141674    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 22:02:37.707035 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.707001    2578 generic.go:356] "Generic (PLEG): container finished" podID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerID="4a2c4ca5e012bf4c0c3098adf82f0c01778f6df30a6a3c404b0d35ec796f6632" exitCode=0
Sep 09 22:02:37.707369 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.707071    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" event={"ID":"787bc763-f469-4eb4-a3cb-bef97d299dd0","Type":"ContainerDied","Data":"4a2c4ca5e012bf4c0c3098adf82f0c01778f6df30a6a3c404b0d35ec796f6632"}
Sep 09 22:02:37.707369 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.707131    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m" event={"ID":"787bc763-f469-4eb4-a3cb-bef97d299dd0","Type":"ContainerDied","Data":"c2be9394283f821a9f660abb40d3201df52287cacf6156b22fd33b90ff55d3f8"}
Sep 09 22:02:37.707369 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.707149    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2be9394283f821a9f660abb40d3201df52287cacf6156b22fd33b90ff55d3f8"
Sep 09 22:02:37.707369 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.707163    2578 scope.go:117] "RemoveContainer" containerID="bebe50c6962b4d169194addd2d75de38b300d8d222a6533f2d7c114473113658"
Sep 09 22:02:37.716223 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.716202    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 22:02:37.816660 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.816634    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hgtj\" (UniqueName: \"kubernetes.io/projected/787bc763-f469-4eb4-a3cb-bef97d299dd0-kube-api-access-5hgtj\") pod \"787bc763-f469-4eb4-a3cb-bef97d299dd0\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") "
Sep 09 22:02:37.816833 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.816667    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-podinfo\") pod \"787bc763-f469-4eb4-a3cb-bef97d299dd0\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") "
Sep 09 22:02:37.816833 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.816693    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-workload-socket\") pod \"787bc763-f469-4eb4-a3cb-bef97d299dd0\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") "
Sep 09 22:02:37.816833 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.816716    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-envoy\") pod \"787bc763-f469-4eb4-a3cb-bef97d299dd0\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") "
Sep 09 22:02:37.816833 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.816759    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-data\") pod \"787bc763-f469-4eb4-a3cb-bef97d299dd0\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") "
Sep 09 22:02:37.816833 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.816788    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/787bc763-f469-4eb4-a3cb-bef97d299dd0-istiod-ca-cert\") pod \"787bc763-f469-4eb4-a3cb-bef97d299dd0\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") "
Sep 09 22:02:37.816833 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.816821    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-token\") pod \"787bc763-f469-4eb4-a3cb-bef97d299dd0\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") "
Sep 09 22:02:37.817199 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.816844    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-credential-socket\") pod \"787bc763-f469-4eb4-a3cb-bef97d299dd0\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") "
Sep 09 22:02:37.817199 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.816894    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-workload-certs\") pod \"787bc763-f469-4eb4-a3cb-bef97d299dd0\" (UID: \"787bc763-f469-4eb4-a3cb-bef97d299dd0\") "
Sep 09 22:02:37.817199 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.817010    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "787bc763-f469-4eb4-a3cb-bef97d299dd0" (UID: "787bc763-f469-4eb4-a3cb-bef97d299dd0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:02:37.817199 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.817081    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-data" (OuterVolumeSpecName: "istio-data") pod "787bc763-f469-4eb4-a3cb-bef97d299dd0" (UID: "787bc763-f469-4eb4-a3cb-bef97d299dd0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:02:37.817422 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.817296    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "787bc763-f469-4eb4-a3cb-bef97d299dd0" (UID: "787bc763-f469-4eb4-a3cb-bef97d299dd0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:02:37.817422 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.817313    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:37.817422 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.817326    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/787bc763-f469-4eb4-a3cb-bef97d299dd0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "787bc763-f469-4eb4-a3cb-bef97d299dd0" (UID: "787bc763-f469-4eb4-a3cb-bef97d299dd0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 22:02:37.817422 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.817334    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:37.817422 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.817384    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "787bc763-f469-4eb4-a3cb-bef97d299dd0" (UID: "787bc763-f469-4eb4-a3cb-bef97d299dd0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:02:37.818960 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.818937    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "787bc763-f469-4eb4-a3cb-bef97d299dd0" (UID: "787bc763-f469-4eb4-a3cb-bef97d299dd0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:02:37.819076 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.818991    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/787bc763-f469-4eb4-a3cb-bef97d299dd0-kube-api-access-5hgtj" (OuterVolumeSpecName: "kube-api-access-5hgtj") pod "787bc763-f469-4eb4-a3cb-bef97d299dd0" (UID: "787bc763-f469-4eb4-a3cb-bef97d299dd0"). InnerVolumeSpecName "kube-api-access-5hgtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:02:37.819147 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.819131    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "787bc763-f469-4eb4-a3cb-bef97d299dd0" (UID: "787bc763-f469-4eb4-a3cb-bef97d299dd0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 22:02:37.819327 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.819308    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-token" (OuterVolumeSpecName: "istio-token") pod "787bc763-f469-4eb4-a3cb-bef97d299dd0" (UID: "787bc763-f469-4eb4-a3cb-bef97d299dd0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:02:37.918790 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.918754    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:37.918790 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.918789    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:37.918790 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.918798    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:37.919001 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.918808    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hgtj\" (UniqueName: \"kubernetes.io/projected/787bc763-f469-4eb4-a3cb-bef97d299dd0-kube-api-access-5hgtj\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:37.919001 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.918817    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:37.919001 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.918825    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/787bc763-f469-4eb4-a3cb-bef97d299dd0-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:37.919001 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:37.918832    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/787bc763-f469-4eb4-a3cb-bef97d299dd0-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:02:38.721843 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:38.721811    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"
Sep 09 22:02:38.755670 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:38.755643    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"]
Sep 09 22:02:38.762017 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:38.761993    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-3fa986-openshift-default-d7f4fdfd6-n8f7m"]
Sep 09 22:02:39.003982 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:39.003878    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" path="/var/lib/kubelet/pods/787bc763-f469-4eb4-a3cb-bef97d299dd0/volumes"
Sep 09 22:02:41.775720 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:41.775687    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:02:52.281150 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:52.281115    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:02:56.152081 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152002    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7495-d26541"]
Sep 09 22:02:56.152437 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152378    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd921041-9f42-4bc7-945c-50cb1f428df5" containerName="main"
Sep 09 22:02:56.152437 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152391    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd921041-9f42-4bc7-945c-50cb1f428df5" containerName="main"
Sep 09 22:02:56.152437 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152405    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48a06290-8726-4a1f-985a-fbfa0923bdb5" containerName="main"
Sep 09 22:02:56.152437 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152411    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="48a06290-8726-4a1f-985a-fbfa0923bdb5" containerName="main"
Sep 09 22:02:56.152437 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152420    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4c49e97-1095-4f2c-a253-89cf2917fd5a" containerName="test-curl-7495-69f2dd"
Sep 09 22:02:56.152437 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152426    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4c49e97-1095-4f2c-a253-89cf2917fd5a" containerName="test-curl-7495-69f2dd"
Sep 09 22:02:56.152437 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152440    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerName="istio-proxy"
Sep 09 22:02:56.152664 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152446    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerName="istio-proxy"
Sep 09 22:02:56.152664 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152459    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerName="istio-proxy"
Sep 09 22:02:56.152664 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152465    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerName="istio-proxy"
Sep 09 22:02:56.152664 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152523    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d783cd0c-d8a3-44c7-b9fd-39c0821873fa" containerName="istio-proxy"
Sep 09 22:02:56.152664 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152534    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd921041-9f42-4bc7-945c-50cb1f428df5" containerName="main"
Sep 09 22:02:56.152664 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152540    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerName="istio-proxy"
Sep 09 22:02:56.152664 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152546    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="48a06290-8726-4a1f-985a-fbfa0923bdb5" containerName="main"
Sep 09 22:02:56.152664 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.152554    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4c49e97-1095-4f2c-a253-89cf2917fd5a" containerName="test-curl-7495-69f2dd"
Sep 09 22:02:56.157099 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.157049    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7495-d26541"
Sep 09 22:02:56.159495 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.159476    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-k4l58\""
Sep 09 22:02:56.160634 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.160612    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7495-d26541"]
Sep 09 22:02:56.268849 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.268815    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8pph\" (UniqueName: \"kubernetes.io/projected/3e4f0e23-3985-41d4-a031-71e73353f849-kube-api-access-q8pph\") pod \"test-curl-7495-d26541\" (UID: \"3e4f0e23-3985-41d4-a031-71e73353f849\") " pod="opendatahub/test-curl-7495-d26541"
Sep 09 22:02:56.369924 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.369898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8pph\" (UniqueName: \"kubernetes.io/projected/3e4f0e23-3985-41d4-a031-71e73353f849-kube-api-access-q8pph\") pod \"test-curl-7495-d26541\" (UID: \"3e4f0e23-3985-41d4-a031-71e73353f849\") " pod="opendatahub/test-curl-7495-d26541"
Sep 09 22:02:56.378761 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.378740    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8pph\" (UniqueName: \"kubernetes.io/projected/3e4f0e23-3985-41d4-a031-71e73353f849-kube-api-access-q8pph\") pod \"test-curl-7495-d26541\" (UID: \"3e4f0e23-3985-41d4-a031-71e73353f849\") " pod="opendatahub/test-curl-7495-d26541"
Sep 09 22:02:56.467859 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.467807    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7495-d26541"
Sep 09 22:02:56.738943 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.738916    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7495-d26541"]
Sep 09 22:02:56.740251 ip-10-0-140-157 kubenswrapper[2578]: W0909 22:02:56.740223    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e4f0e23_3985_41d4_a031_71e73353f849.slice/crio-89300bcde9c97bed4f595397fc60d750b4cf0298c9569320dbb643df45a4d34f WatchSource:0}: Error finding container 89300bcde9c97bed4f595397fc60d750b4cf0298c9569320dbb643df45a4d34f: Status 404 returned error can't find the container with id 89300bcde9c97bed4f595397fc60d750b4cf0298c9569320dbb643df45a4d34f
Sep 09 22:02:56.829504 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.829480    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7495-d26541" event={"ID":"3e4f0e23-3985-41d4-a031-71e73353f849","Type":"ContainerStarted","Data":"5cd5f989bc587ee6e148744817c5298754893c9d515400a9587c97d4a26c0f94"}
Sep 09 22:02:56.829647 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.829518    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7495-d26541" event={"ID":"3e4f0e23-3985-41d4-a031-71e73353f849","Type":"ContainerStarted","Data":"89300bcde9c97bed4f595397fc60d750b4cf0298c9569320dbb643df45a4d34f"}
Sep 09 22:02:56.846045 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:56.846003    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7495-d26541" podStartSLOduration=0.845989471 podStartE2EDuration="845.989471ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:02:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:02:56.843365033 +0000 UTC m=+1796.316329943" watchObservedRunningTime="2026-09-09 22:02:56.845989471 +0000 UTC m=+1796.318954398"
Sep 09 22:02:57.439341 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:57.439294    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7495-d26541"]
Sep 09 22:02:57.445480 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:57.445452    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7495-d26541"]
Sep 09 22:02:57.558711 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:57.558678    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7495-181337"]
Sep 09 22:02:57.559129 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:57.559114    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e4f0e23-3985-41d4-a031-71e73353f849" containerName="test-curl-7495-d26541"
Sep 09 22:02:57.559176 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:57.559130    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e4f0e23-3985-41d4-a031-71e73353f849" containerName="test-curl-7495-d26541"
Sep 09 22:02:57.559219 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:57.559192    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e4f0e23-3985-41d4-a031-71e73353f849" containerName="test-curl-7495-d26541"
Sep 09 22:02:57.559219 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:57.559202    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="787bc763-f469-4eb4-a3cb-bef97d299dd0" containerName="istio-proxy"
Sep 09 22:02:57.562330 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:57.562313    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7495-181337"
Sep 09 22:02:57.567458 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:57.567435    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7495-181337"]
Sep 09 22:02:57.580683 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:57.580662    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn9pf\" (UniqueName: \"kubernetes.io/projected/23d7a243-8675-4279-a1a1-d9d9ebd879b8-kube-api-access-cn9pf\") pod \"test-curl-7495-181337\" (UID: \"23d7a243-8675-4279-a1a1-d9d9ebd879b8\") " pod="opendatahub/test-curl-7495-181337"
Sep 09 22:02:57.681437 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:57.681411    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn9pf\" (UniqueName: \"kubernetes.io/projected/23d7a243-8675-4279-a1a1-d9d9ebd879b8-kube-api-access-cn9pf\") pod \"test-curl-7495-181337\" (UID: \"23d7a243-8675-4279-a1a1-d9d9ebd879b8\") " pod="opendatahub/test-curl-7495-181337"
Sep 09 22:02:57.689725 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:57.689662    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn9pf\" (UniqueName: \"kubernetes.io/projected/23d7a243-8675-4279-a1a1-d9d9ebd879b8-kube-api-access-cn9pf\") pod \"test-curl-7495-181337\" (UID: \"23d7a243-8675-4279-a1a1-d9d9ebd879b8\") " pod="opendatahub/test-curl-7495-181337"
Sep 09 22:02:57.836641 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:02:57.836610    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7495-d26541" container="test-curl-7495-d26541"
Sep 09 22:02:57.873991 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:57.873945    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7495-181337"
Sep 09 22:02:58.145354 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:58.145323    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7495-181337"]
Sep 09 22:02:58.147709 ip-10-0-140-157 kubenswrapper[2578]: W0909 22:02:58.147661    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23d7a243_8675_4279_a1a1_d9d9ebd879b8.slice/crio-a129ea9be2d9a8239db7c6137b471ff59563013b08daa7896b772568b095bef7 WatchSource:0}: Error finding container a129ea9be2d9a8239db7c6137b471ff59563013b08daa7896b772568b095bef7: Status 404 returned error can't find the container with id a129ea9be2d9a8239db7c6137b471ff59563013b08daa7896b772568b095bef7
Sep 09 22:02:58.844743 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:58.844711    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7495-181337" event={"ID":"23d7a243-8675-4279-a1a1-d9d9ebd879b8","Type":"ContainerStarted","Data":"99d067ac8ca260981c6bf67143f44629888e93dbffc009923e73d78b1f9bfdaf"}
Sep 09 22:02:58.845169 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:58.844754    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7495-181337" event={"ID":"23d7a243-8675-4279-a1a1-d9d9ebd879b8","Type":"ContainerStarted","Data":"a129ea9be2d9a8239db7c6137b471ff59563013b08daa7896b772568b095bef7"}
Sep 09 22:02:58.845169 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:58.844856    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7495-d26541" podUID="3e4f0e23-3985-41d4-a031-71e73353f849" containerName="test-curl-7495-d26541" containerID="cri-o://5cd5f989bc587ee6e148744817c5298754893c9d515400a9587c97d4a26c0f94" gracePeriod=2
Sep 09 22:02:58.847003 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:02:58.846978    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7495-d26541\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="3e4f0e23-3985-41d4-a031-71e73353f849" pod="opendatahub/test-curl-7495-d26541"
Sep 09 22:02:58.863843 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:02:58.863816    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7495-d26541\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="3e4f0e23-3985-41d4-a031-71e73353f849" pod="opendatahub/test-curl-7495-d26541"
Sep 09 22:02:58.865015 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:58.864979    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7495-181337" podStartSLOduration=1.8649546529999999 podStartE2EDuration="1.864954653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:02:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:02:58.861916073 +0000 UTC m=+1798.334880983" watchObservedRunningTime="2026-09-09 22:02:58.864954653 +0000 UTC m=+1798.337919563"
Sep 09 22:02:59.449878 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:59.449841    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7495-181337"]
Sep 09 22:02:59.453388 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:02:59.453358    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7495-181337"]
Sep 09 22:02:59.850575 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:02:59.850545    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7495-181337" container="test-curl-7495-181337"
Sep 09 22:03:00.855742 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:00.855709    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7495-181337" podUID="23d7a243-8675-4279-a1a1-d9d9ebd879b8" containerName="test-curl-7495-181337" containerID="cri-o://99d067ac8ca260981c6bf67143f44629888e93dbffc009923e73d78b1f9bfdaf" gracePeriod=2
Sep 09 22:03:00.857756 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:03:00.857718    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7495-181337\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="23d7a243-8675-4279-a1a1-d9d9ebd879b8" pod="opendatahub/test-curl-7495-181337"
Sep 09 22:03:01.004343 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:03:01.004314    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7495-181337\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="23d7a243-8675-4279-a1a1-d9d9ebd879b8" pod="opendatahub/test-curl-7495-181337"
Sep 09 22:03:01.029929 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:01.029906    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7495-d26541"
Sep 09 22:03:01.031625 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:03:01.031605    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7495-d26541\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="3e4f0e23-3985-41d4-a031-71e73353f849" pod="opendatahub/test-curl-7495-d26541"
Sep 09 22:03:01.110820 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:01.110746    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8pph\" (UniqueName: \"kubernetes.io/projected/3e4f0e23-3985-41d4-a031-71e73353f849-kube-api-access-q8pph\") pod \"3e4f0e23-3985-41d4-a031-71e73353f849\" (UID: \"3e4f0e23-3985-41d4-a031-71e73353f849\") "
Sep 09 22:03:01.112866 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:01.112842    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e4f0e23-3985-41d4-a031-71e73353f849-kube-api-access-q8pph" (OuterVolumeSpecName: "kube-api-access-q8pph") pod "3e4f0e23-3985-41d4-a031-71e73353f849" (UID: "3e4f0e23-3985-41d4-a031-71e73353f849"). InnerVolumeSpecName "kube-api-access-q8pph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:03:01.160799 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:01.160773    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 22:03:01.164844 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:01.164822    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 22:03:01.212155 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:01.212136    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8pph\" (UniqueName: \"kubernetes.io/projected/3e4f0e23-3985-41d4-a031-71e73353f849-kube-api-access-q8pph\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:03:01.414248 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:01.414182    2578 scope.go:117] "RemoveContainer" containerID="4a2c4ca5e012bf4c0c3098adf82f0c01778f6df30a6a3c404b0d35ec796f6632"
Sep 09 22:03:01.862797 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:01.862757    2578 generic.go:356] "Generic (PLEG): container finished" podID="3e4f0e23-3985-41d4-a031-71e73353f849" containerID="5cd5f989bc587ee6e148744817c5298754893c9d515400a9587c97d4a26c0f94" exitCode=137
Sep 09 22:03:01.863228 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:01.862813    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7495-d26541"
Sep 09 22:03:01.863228 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:01.862857    2578 scope.go:117] "RemoveContainer" containerID="5cd5f989bc587ee6e148744817c5298754893c9d515400a9587c97d4a26c0f94"
Sep 09 22:03:01.865037 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:03:01.865010    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7495-d26541\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="3e4f0e23-3985-41d4-a031-71e73353f849" pod="opendatahub/test-curl-7495-d26541"
Sep 09 22:03:01.871874 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:01.871859    2578 scope.go:117] "RemoveContainer" containerID="5cd5f989bc587ee6e148744817c5298754893c9d515400a9587c97d4a26c0f94"
Sep 09 22:03:01.872161 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:03:01.872140    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cd5f989bc587ee6e148744817c5298754893c9d515400a9587c97d4a26c0f94\": container with ID starting with 5cd5f989bc587ee6e148744817c5298754893c9d515400a9587c97d4a26c0f94 not found: ID does not exist" containerID="5cd5f989bc587ee6e148744817c5298754893c9d515400a9587c97d4a26c0f94"
Sep 09 22:03:01.872227 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:01.872167    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cd5f989bc587ee6e148744817c5298754893c9d515400a9587c97d4a26c0f94"} err="failed to get container status \"5cd5f989bc587ee6e148744817c5298754893c9d515400a9587c97d4a26c0f94\": rpc error: code = NotFound desc = could not find container \"5cd5f989bc587ee6e148744817c5298754893c9d515400a9587c97d4a26c0f94\": container with ID starting with 5cd5f989bc587ee6e148744817c5298754893c9d515400a9587c97d4a26c0f94 not found: ID does not exist"
Sep 09 22:03:01.872839 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:03:01.872814    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7495-d26541\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="3e4f0e23-3985-41d4-a031-71e73353f849" pod="opendatahub/test-curl-7495-d26541"
Sep 09 22:03:03.004106 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:03.004076    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e4f0e23-3985-41d4-a031-71e73353f849" path="/var/lib/kubelet/pods/3e4f0e23-3985-41d4-a031-71e73353f849/volumes"
Sep 09 22:03:03.049058 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:03.049037    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7495-181337"
Sep 09 22:03:03.126549 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:03.126480    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn9pf\" (UniqueName: \"kubernetes.io/projected/23d7a243-8675-4279-a1a1-d9d9ebd879b8-kube-api-access-cn9pf\") pod \"23d7a243-8675-4279-a1a1-d9d9ebd879b8\" (UID: \"23d7a243-8675-4279-a1a1-d9d9ebd879b8\") "
Sep 09 22:03:03.128504 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:03.128477    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23d7a243-8675-4279-a1a1-d9d9ebd879b8-kube-api-access-cn9pf" (OuterVolumeSpecName: "kube-api-access-cn9pf") pod "23d7a243-8675-4279-a1a1-d9d9ebd879b8" (UID: "23d7a243-8675-4279-a1a1-d9d9ebd879b8"). InnerVolumeSpecName "kube-api-access-cn9pf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:03:03.227220 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:03.227188    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn9pf\" (UniqueName: \"kubernetes.io/projected/23d7a243-8675-4279-a1a1-d9d9ebd879b8-kube-api-access-cn9pf\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:03:03.879142 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:03.879107    2578 generic.go:356] "Generic (PLEG): container finished" podID="23d7a243-8675-4279-a1a1-d9d9ebd879b8" containerID="99d067ac8ca260981c6bf67143f44629888e93dbffc009923e73d78b1f9bfdaf" exitCode=137
Sep 09 22:03:03.879298 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:03.879146    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7495-181337"
Sep 09 22:03:03.879298 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:03.879209    2578 scope.go:117] "RemoveContainer" containerID="99d067ac8ca260981c6bf67143f44629888e93dbffc009923e73d78b1f9bfdaf"
Sep 09 22:03:03.888505 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:03.888488    2578 scope.go:117] "RemoveContainer" containerID="99d067ac8ca260981c6bf67143f44629888e93dbffc009923e73d78b1f9bfdaf"
Sep 09 22:03:03.888767 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:03:03.888750    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99d067ac8ca260981c6bf67143f44629888e93dbffc009923e73d78b1f9bfdaf\": container with ID starting with 99d067ac8ca260981c6bf67143f44629888e93dbffc009923e73d78b1f9bfdaf not found: ID does not exist" containerID="99d067ac8ca260981c6bf67143f44629888e93dbffc009923e73d78b1f9bfdaf"
Sep 09 22:03:03.888830 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:03.888778    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99d067ac8ca260981c6bf67143f44629888e93dbffc009923e73d78b1f9bfdaf"} err="failed to get container status \"99d067ac8ca260981c6bf67143f44629888e93dbffc009923e73d78b1f9bfdaf\": rpc error: code = NotFound desc = could not find container \"99d067ac8ca260981c6bf67143f44629888e93dbffc009923e73d78b1f9bfdaf\": container with ID starting with 99d067ac8ca260981c6bf67143f44629888e93dbffc009923e73d78b1f9bfdaf not found: ID does not exist"
Sep 09 22:03:03.889117 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:03:03.889095    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7495-181337\" is forbidden: User \"system:node:ip-10-0-140-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-157.ec2.internal' and this object" podUID="23d7a243-8675-4279-a1a1-d9d9ebd879b8" pod="opendatahub/test-curl-7495-181337"
Sep 09 22:03:05.002737 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:05.002703    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23d7a243-8675-4279-a1a1-d9d9ebd879b8" path="/var/lib/kubelet/pods/23d7a243-8675-4279-a1a1-d9d9ebd879b8/volumes"
Sep 09 22:03:06.368688 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:06.368658    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:03:17.973338 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:17.973308    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:03:56.982626 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:03:56.982590    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:04:17.573787 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:17.573750    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:04:27.381275 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:27.381197    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:04:35.596813 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.596782    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"]
Sep 09 22:04:35.597324 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.597304    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23d7a243-8675-4279-a1a1-d9d9ebd879b8" containerName="test-curl-7495-181337"
Sep 09 22:04:35.597374 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.597329    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="23d7a243-8675-4279-a1a1-d9d9ebd879b8" containerName="test-curl-7495-181337"
Sep 09 22:04:35.597438 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.597427    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="23d7a243-8675-4279-a1a1-d9d9ebd879b8" containerName="test-curl-7495-181337"
Sep 09 22:04:35.600680 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.600662    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.605548 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.605527    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-100032-openshift-default-dockercfg-xnfmh\""
Sep 09 22:04:35.615618 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.615598    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"]
Sep 09 22:04:35.690553 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.690521    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-workload-socket\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.690710 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.690615    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-data\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.690710 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.690686    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-token\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.690824 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.690742    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istiod-ca-cert\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.690824 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.690786    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-credential-socket\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.690936 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.690861    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-workload-certs\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.690936 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.690926    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-envoy\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.691061 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.690987    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-podinfo\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.691061 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.691035    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbq88\" (UniqueName: \"kubernetes.io/projected/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-kube-api-access-sbq88\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.792297 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.792261    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbq88\" (UniqueName: \"kubernetes.io/projected/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-kube-api-access-sbq88\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.792483 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.792317    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-workload-socket\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.792483 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.792335    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-data\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.792483 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.792361    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-token\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.792483 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.792385    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istiod-ca-cert\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.792483 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.792408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-credential-socket\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.792483 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.792439    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-workload-certs\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.792483 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.792479    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-envoy\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.792855 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.792518    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-podinfo\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.792855 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.792750    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-workload-socket\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.792855 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.792795    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-credential-socket\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.793002 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.792852    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-data\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.793002 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.792949    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-workload-certs\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.793312 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.793294    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istiod-ca-cert\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.794813 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.794790    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-envoy\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.795122 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.795103    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-podinfo\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.799729 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.799706    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbq88\" (UniqueName: \"kubernetes.io/projected/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-kube-api-access-sbq88\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.799729 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.799717    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-token\") pod \"e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") " pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:35.910896 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:35.910833    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:36.195606 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:36.195574    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"]
Sep 09 22:04:36.197388 ip-10-0-140-157 kubenswrapper[2578]: W0909 22:04:36.197366    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4397d91e_a780_4e4d_80a5_a58a2cbcef2f.slice/crio-5b34c7775c64b7f6dc98ea55b0a19e1cb4b875d30f96639b888e9bec44cb42bb WatchSource:0}: Error finding container 5b34c7775c64b7f6dc98ea55b0a19e1cb4b875d30f96639b888e9bec44cb42bb: Status 404 returned error can't find the container with id 5b34c7775c64b7f6dc98ea55b0a19e1cb4b875d30f96639b888e9bec44cb42bb
Sep 09 22:04:36.199515 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:36.199482    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:04:36.199608 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:36.199546    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:04:36.199608 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:36.199575    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:04:36.384026 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:36.383991    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:04:36.430946 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:36.430917    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8" event={"ID":"4397d91e-a780-4e4d-80a5-a58a2cbcef2f","Type":"ContainerStarted","Data":"4e6d5494a6d871ec5fea58eeee9459608e05dd5746c43878b2c209e727ca794a"}
Sep 09 22:04:36.431126 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:36.430992    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8" event={"ID":"4397d91e-a780-4e4d-80a5-a58a2cbcef2f","Type":"ContainerStarted","Data":"5b34c7775c64b7f6dc98ea55b0a19e1cb4b875d30f96639b888e9bec44cb42bb"}
Sep 09 22:04:36.452811 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:36.452710    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8" podStartSLOduration=1.452693984 podStartE2EDuration="1.452693984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:04:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:04:36.451361096 +0000 UTC m=+1895.924326005" watchObservedRunningTime="2026-09-09 22:04:36.452693984 +0000 UTC m=+1895.925658893"
Sep 09 22:04:36.911889 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:36.911850    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:37.916567 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:37.916537    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:38.444391 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:38.444363    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:38.445803 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:38.445779    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:48.269797 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:48.269761    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:04:53.445636 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:53.445603    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 09 22:04:53.446073 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:53.445654    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8" podUID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 09 22:04:53.533066 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:53.533038    2578 generic.go:356] "Generic (PLEG): container finished" podID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerID="4e6d5494a6d871ec5fea58eeee9459608e05dd5746c43878b2c209e727ca794a" exitCode=0
Sep 09 22:04:53.533199 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:53.533091    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8" event={"ID":"4397d91e-a780-4e4d-80a5-a58a2cbcef2f","Type":"ContainerDied","Data":"4e6d5494a6d871ec5fea58eeee9459608e05dd5746c43878b2c209e727ca794a"}
Sep 09 22:04:53.533474 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:53.533459    2578 scope.go:117] "RemoveContainer" containerID="4e6d5494a6d871ec5fea58eeee9459608e05dd5746c43878b2c209e727ca794a"
Sep 09 22:04:53.534564 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:53.534531    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:04:53.534655 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:53.534614    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:04:53.534655 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:53.534647    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:04:54.559962 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:54.559912    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8" event={"ID":"4397d91e-a780-4e4d-80a5-a58a2cbcef2f","Type":"ContainerStarted","Data":"ff41572f0c31e947dc64f2f509cb4fd88cb134f331a857bc012a6ad2a22ff8cc"}
Sep 09 22:04:54.911781 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:54.911684    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:54.911992 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:54.911946    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 09 22:04:54.912050 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:54.912022    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8" podUID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 09 22:04:55.911910 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:55.911874    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 09 22:04:55.912313 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:55.911928    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8" podUID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 09 22:04:56.911850 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:56.911808    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 09 22:04:56.912059 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:56.911875    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8" podUID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 09 22:04:57.911561 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:57.911520    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 09 22:04:57.911769 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:57.911583    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8" podUID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 09 22:04:58.911952 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:58.911916    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Sep 09 22:04:58.912365 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:58.912008    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8" podUID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Sep 09 22:04:59.915834 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:59.915801    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:59.916227 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:59.916201    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:04:59.916890 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:04:59.916875    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:05:04.391067 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.391033    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"]
Sep 09 22:05:04.393945 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.393918    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.396172 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.396148    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-855e86-openshift-default-dockercfg-mcmp7\""
Sep 09 22:05:04.405672 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.405645    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"]
Sep 09 22:05:04.462601 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.462565    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-workload-certs\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.462739 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.462632    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-workload-socket\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.462739 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.462664    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99q2q\" (UniqueName: \"kubernetes.io/projected/5ba01acc-614c-48e1-b608-9adbef027488-kube-api-access-99q2q\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.462739 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.462714    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-istio-envoy\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.462908 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.462745    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ba01acc-614c-48e1-b608-9adbef027488-istio-token\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.462908 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.462840    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-istio-data\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.462908 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.462875    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ba01acc-614c-48e1-b608-9adbef027488-istiod-ca-cert\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.463082 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.462926    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-credential-socket\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.463082 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.463000    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ba01acc-614c-48e1-b608-9adbef027488-istio-podinfo\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.563732 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.563697    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-istio-envoy\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.563732 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.563734    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ba01acc-614c-48e1-b608-9adbef027488-istio-token\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.563978 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.563778    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-istio-data\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.563978 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.563797    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ba01acc-614c-48e1-b608-9adbef027488-istiod-ca-cert\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.563978 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.563824    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-credential-socket\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.563978 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.563858    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ba01acc-614c-48e1-b608-9adbef027488-istio-podinfo\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.563978 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.563910    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-workload-certs\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.563978 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.563938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-workload-socket\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.564344 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.563961    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99q2q\" (UniqueName: \"kubernetes.io/projected/5ba01acc-614c-48e1-b608-9adbef027488-kube-api-access-99q2q\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.564344 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.564252    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-istio-data\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.564445 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.564401    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-credential-socket\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.564493 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.564435    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-workload-certs\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.564552 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.564533    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-workload-socket\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.564693 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.564671    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ba01acc-614c-48e1-b608-9adbef027488-istiod-ca-cert\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.566105 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.566079    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-istio-envoy\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.566468 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.566445    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ba01acc-614c-48e1-b608-9adbef027488-istio-podinfo\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.572120 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.572095    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ba01acc-614c-48e1-b608-9adbef027488-istio-token\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.572326 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.572307    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99q2q\" (UniqueName: \"kubernetes.io/projected/5ba01acc-614c-48e1-b608-9adbef027488-kube-api-access-99q2q\") pod \"e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") " pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.705328 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.705264    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:04.981542 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.981507    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"]
Sep 09 22:05:04.982424 ip-10-0-140-157 kubenswrapper[2578]: W0909 22:05:04.982401    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba01acc_614c_48e1_b608_9adbef027488.slice/crio-534343a2d54cf986cf72048ddeb9e3aabdf8209f85c08501f29e557afe25aca6 WatchSource:0}: Error finding container 534343a2d54cf986cf72048ddeb9e3aabdf8209f85c08501f29e557afe25aca6: Status 404 returned error can't find the container with id 534343a2d54cf986cf72048ddeb9e3aabdf8209f85c08501f29e557afe25aca6
Sep 09 22:05:04.984693 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.984659    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:05:04.984850 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.984725    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:05:04.984850 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:04.984757    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:05:05.628897 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:05.628860    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5" event={"ID":"5ba01acc-614c-48e1-b608-9adbef027488","Type":"ContainerStarted","Data":"1f6637178dcd4cbfaa3a2c70bf2451e31a1ee9a38ce72f864cc5dcd96f299a62"}
Sep 09 22:05:05.629350 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:05.628910    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5" event={"ID":"5ba01acc-614c-48e1-b608-9adbef027488","Type":"ContainerStarted","Data":"534343a2d54cf986cf72048ddeb9e3aabdf8209f85c08501f29e557afe25aca6"}
Sep 09 22:05:05.649149 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:05.649103    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5" podStartSLOduration=1.649089376 podStartE2EDuration="1.649089376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:05:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:05:05.646866429 +0000 UTC m=+1925.119831339" watchObservedRunningTime="2026-09-09 22:05:05.649089376 +0000 UTC m=+1925.122054286"
Sep 09 22:05:05.705772 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:05.705749    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:06.711471 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:06.711440    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:07.641191 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:07.641167    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:07.642027 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:07.642006    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:15.686806 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:15.686770    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:05:23.682165 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:23.682114    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 22:05:23.682694 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:23.682213    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5" podUID="5ba01acc-614c-48e1-b608-9adbef027488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 22:05:25.755902 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:25.755798    2578 generic.go:356] "Generic (PLEG): container finished" podID="5ba01acc-614c-48e1-b608-9adbef027488" containerID="1f6637178dcd4cbfaa3a2c70bf2451e31a1ee9a38ce72f864cc5dcd96f299a62" exitCode=0
Sep 09 22:05:25.755902 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:25.755880    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5" event={"ID":"5ba01acc-614c-48e1-b608-9adbef027488","Type":"ContainerDied","Data":"1f6637178dcd4cbfaa3a2c70bf2451e31a1ee9a38ce72f864cc5dcd96f299a62"}
Sep 09 22:05:25.756403 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:25.756310    2578 scope.go:117] "RemoveContainer" containerID="1f6637178dcd4cbfaa3a2c70bf2451e31a1ee9a38ce72f864cc5dcd96f299a62"
Sep 09 22:05:25.757405 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:25.757377    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:05:25.757473 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:25.757445    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:05:25.757514 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:25.757475    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:05:26.763307 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:26.763271    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5" event={"ID":"5ba01acc-614c-48e1-b608-9adbef027488","Type":"ContainerStarted","Data":"d9cf05e64f2d72e4e938550657252a8da91168f7f6fce5b25a33be58591b5817"}
Sep 09 22:05:27.705767 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:27.705727    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:27.705952 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:27.705902    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Sep 09 22:05:27.706048 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:27.705949    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5" podUID="5ba01acc-614c-48e1-b608-9adbef027488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Sep 09 22:05:28.706188 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:28.706150    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Sep 09 22:05:28.706680 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:28.706207    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5" podUID="5ba01acc-614c-48e1-b608-9adbef027488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Sep 09 22:05:29.705845 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:29.705808    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Sep 09 22:05:29.706070 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:29.705873    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5" podUID="5ba01acc-614c-48e1-b608-9adbef027488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Sep 09 22:05:30.706516 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:30.706482    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Sep 09 22:05:30.706942 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:30.706537    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5" podUID="5ba01acc-614c-48e1-b608-9adbef027488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Sep 09 22:05:31.706381 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:31.706345    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Sep 09 22:05:31.706567 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:31.706405    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5" podUID="5ba01acc-614c-48e1-b608-9adbef027488" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Sep 09 22:05:32.709777 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:32.709743    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:32.710176 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:32.710120    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:32.710869 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:32.710845    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:05:47.883232 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:05:47.883188    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:06:17.969740 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:17.969702    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:06:25.072423 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.072381    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"]
Sep 09 22:06:25.075191 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.075169    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.077633 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.077609    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-bcb351-openshift-default-dockercfg-6w9qf\""
Sep 09 22:06:25.087074 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.087052    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"]
Sep 09 22:06:25.204371 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.204334    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0cf0830-d201-4808-a84d-a026c51c36f4-istiod-ca-cert\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.204624 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.204394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-data\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.204624 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.204460    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-podinfo\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.204624 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.204493    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-token\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.204624 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.204532    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7msp5\" (UniqueName: \"kubernetes.io/projected/a0cf0830-d201-4808-a84d-a026c51c36f4-kube-api-access-7msp5\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.204624 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.204577    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-credential-socket\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.204624 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.204596    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-workload-certs\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.204873 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.204669    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-workload-socket\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.204873 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.204710    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-envoy\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.305860 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.305834    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7msp5\" (UniqueName: \"kubernetes.io/projected/a0cf0830-d201-4808-a84d-a026c51c36f4-kube-api-access-7msp5\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.306035 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.305870    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-credential-socket\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.306035 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.305887    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-workload-certs\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.306035 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.305917    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-workload-socket\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.306212 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.306039    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-envoy\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.306212 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.306087    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0cf0830-d201-4808-a84d-a026c51c36f4-istiod-ca-cert\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.306212 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.306142    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-data\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.306212 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.306208    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-podinfo\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.306397 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.306238    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-token\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.306397 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.306296    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-credential-socket\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.306397 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.306332    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-workload-socket\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.306570 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.306500    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-workload-certs\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.306610 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.306593    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-data\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.306775 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.306747    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0cf0830-d201-4808-a84d-a026c51c36f4-istiod-ca-cert\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.308544 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.308521    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-envoy\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.308732 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.308712    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-podinfo\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.314132 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.314103    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-token\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.314373 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.314352    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7msp5\" (UniqueName: \"kubernetes.io/projected/a0cf0830-d201-4808-a84d-a026c51c36f4-kube-api-access-7msp5\") pod \"e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") " pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.387074 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.386955    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:25.669079 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.669051    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"]
Sep 09 22:06:25.669851 ip-10-0-140-157 kubenswrapper[2578]: W0909 22:06:25.669819    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0cf0830_d201_4808_a84d_a026c51c36f4.slice/crio-54c0e7371a48e49a2183d38e2e87a45e0ba1efb206e3005acee3de314321ffc8 WatchSource:0}: Error finding container 54c0e7371a48e49a2183d38e2e87a45e0ba1efb206e3005acee3de314321ffc8: Status 404 returned error can't find the container with id 54c0e7371a48e49a2183d38e2e87a45e0ba1efb206e3005acee3de314321ffc8
Sep 09 22:06:25.672254 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.672218    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:06:25.672366 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.672288    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:06:25.672366 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:25.672320    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:06:26.107046 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:26.107008    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr" event={"ID":"a0cf0830-d201-4808-a84d-a026c51c36f4","Type":"ContainerStarted","Data":"5f41009adb83abb3bd6325f9133979e20db0fd44d62bfc7efef5862bab299ed7"}
Sep 09 22:06:26.107479 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:26.107063    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr" event={"ID":"a0cf0830-d201-4808-a84d-a026c51c36f4","Type":"ContainerStarted","Data":"54c0e7371a48e49a2183d38e2e87a45e0ba1efb206e3005acee3de314321ffc8"}
Sep 09 22:06:26.130552 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:26.130508    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr" podStartSLOduration=1.130493049 podStartE2EDuration="1.130493049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:06:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:06:26.127645891 +0000 UTC m=+2005.600610801" watchObservedRunningTime="2026-09-09 22:06:26.130493049 +0000 UTC m=+2005.603457959"
Sep 09 22:06:26.387701 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:26.387627    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:26.392861 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:26.392835    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:27.112644 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:27.112616    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:27.114020 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:27.114000    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:29.384844 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.384813    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"]
Sep 09 22:06:29.388861 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.388841    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:29.391486 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.391457    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-bcb351-serving-cert\""
Sep 09 22:06:29.399330 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.399308    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"]
Sep 09 22:06:29.443193 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.443161    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:29.443334 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.443235    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:29.443334 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.443297    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6rwd\" (UniqueName: \"kubernetes.io/projected/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-kube-api-access-d6rwd\") pod \"maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:29.443334 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.443320    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:29.544147 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.544114    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:29.544414 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.544184    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:29.544414 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.544257    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6rwd\" (UniqueName: \"kubernetes.io/projected/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-kube-api-access-d6rwd\") pod \"maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:29.544414 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.544288    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:29.544872 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.544848    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:29.546675 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.546653    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:29.546822 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.546802    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:29.561937 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.561918    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6rwd\" (UniqueName: \"kubernetes.io/projected/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-kube-api-access-d6rwd\") pod \"maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:29.700850 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.700762    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:29.985532 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.985499    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"]
Sep 09 22:06:29.987030 ip-10-0-140-157 kubenswrapper[2578]: W0909 22:06:29.986873    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc42ee15_ce0d_45a0_b6da_4f3a7b5d78e8.slice/crio-f11c05387ca356617149115719029b61dd04c0df70369ee3f1d83b306b49c6bb WatchSource:0}: Error finding container f11c05387ca356617149115719029b61dd04c0df70369ee3f1d83b306b49c6bb: Status 404 returned error can't find the container with id f11c05387ca356617149115719029b61dd04c0df70369ee3f1d83b306b49c6bb
Sep 09 22:06:29.988703 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:29.988684    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 22:06:30.132753 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:30.132721    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5" event={"ID":"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8","Type":"ContainerStarted","Data":"f11c05387ca356617149115719029b61dd04c0df70369ee3f1d83b306b49c6bb"}
Sep 09 22:06:31.140634 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:31.140604    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5" event={"ID":"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8","Type":"ContainerStarted","Data":"479413109ee04cd9fabcba227f65f4d48654bbc8802e1fa8722e7d3a43783421"}
Sep 09 22:06:31.141026 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:31.140728    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:31.159576 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:31.159527    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5" podStartSLOduration=1.817042101 podStartE2EDuration="2.159513042s" totalImagesPullingTime="342.470941ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:06:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:06:31.156614741 +0000 UTC m=+2010.629579651" watchObservedRunningTime="2026-09-09 22:06:31.159513042 +0000 UTC m=+2010.632477953"
Sep 09 22:06:36.473525 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:36.473480    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:06:37.154985 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:37.154932    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:06:46.231119 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:46.231028    2578 generic.go:356] "Generic (PLEG): container finished" podID="a0cf0830-d201-4808-a84d-a026c51c36f4" containerID="5f41009adb83abb3bd6325f9133979e20db0fd44d62bfc7efef5862bab299ed7" exitCode=0
Sep 09 22:06:46.231119 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:46.231101    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr" event={"ID":"a0cf0830-d201-4808-a84d-a026c51c36f4","Type":"ContainerDied","Data":"5f41009adb83abb3bd6325f9133979e20db0fd44d62bfc7efef5862bab299ed7"}
Sep 09 22:06:46.231567 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:46.231476    2578 scope.go:117] "RemoveContainer" containerID="5f41009adb83abb3bd6325f9133979e20db0fd44d62bfc7efef5862bab299ed7"
Sep 09 22:06:46.232685 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:46.232648    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:06:46.232767 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:46.232726    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:06:46.232767 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:46.232756    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 22:06:47.244613 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:47.244559    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr" event={"ID":"a0cf0830-d201-4808-a84d-a026c51c36f4","Type":"ContainerStarted","Data":"1b7460d7007313951b80ab1c054d62b51da50043171279bbb6283c837fe151b2"}
Sep 09 22:06:47.387713 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:47.387679    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:47.387990 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:47.387945    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 09 22:06:47.388044 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:47.388019    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr" podUID="a0cf0830-d201-4808-a84d-a026c51c36f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 09 22:06:48.388030 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:48.387992    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 09 22:06:48.388535 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:48.388051    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr" podUID="a0cf0830-d201-4808-a84d-a026c51c36f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 09 22:06:49.388258 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:49.388222    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 09 22:06:49.388733 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:49.388278    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr" podUID="a0cf0830-d201-4808-a84d-a026c51c36f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 09 22:06:50.387807 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:50.387772    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 09 22:06:50.388020 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:50.387831    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr" podUID="a0cf0830-d201-4808-a84d-a026c51c36f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 09 22:06:51.387508 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:51.387471    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 09 22:06:51.387891 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:51.387535    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr" podUID="a0cf0830-d201-4808-a84d-a026c51c36f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 09 22:06:52.404207 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:52.404177    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:52.404636 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:52.404550    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:52.405311 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:52.405290    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:06:59.835804 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:59.835761    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"]
Sep 09 22:06:59.836305 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:06:59.835979    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5" podUID="cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8" containerName="maas-api" containerID="cri-o://479413109ee04cd9fabcba227f65f4d48654bbc8802e1fa8722e7d3a43783421" gracePeriod=30
Sep 09 22:07:00.115362 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.115285    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:07:00.158407 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.158378    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-openshift-service-ca\") pod \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") "
Sep 09 22:07:00.158581 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.158431    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-metrics-certs\") pod \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") "
Sep 09 22:07:00.158581 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.158455    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6rwd\" (UniqueName: \"kubernetes.io/projected/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-kube-api-access-d6rwd\") pod \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") "
Sep 09 22:07:00.158581 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.158496    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-maas-api-tls\") pod \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\" (UID: \"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8\") "
Sep 09 22:07:00.158750 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.158690    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8" (UID: "cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 22:07:00.159014 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.158980    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-openshift-service-ca\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:07:00.160591 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.160570    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8" (UID: "cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 22:07:00.161043 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.161017    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8" (UID: "cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 22:07:00.161137 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.161050    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-kube-api-access-d6rwd" (OuterVolumeSpecName: "kube-api-access-d6rwd") pod "cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8" (UID: "cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8"). InnerVolumeSpecName "kube-api-access-d6rwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:07:00.259729 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.259693    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-metrics-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:07:00.259729 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.259725    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6rwd\" (UniqueName: \"kubernetes.io/projected/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-kube-api-access-d6rwd\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:07:00.259729 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.259735    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8-maas-api-tls\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:07:00.336025 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.335986    2578 generic.go:356] "Generic (PLEG): container finished" podID="cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8" containerID="479413109ee04cd9fabcba227f65f4d48654bbc8802e1fa8722e7d3a43783421" exitCode=0
Sep 09 22:07:00.336208 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.336048    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"
Sep 09 22:07:00.336208 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.336054    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5" event={"ID":"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8","Type":"ContainerDied","Data":"479413109ee04cd9fabcba227f65f4d48654bbc8802e1fa8722e7d3a43783421"}
Sep 09 22:07:00.336208 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.336106    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5" event={"ID":"cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8","Type":"ContainerDied","Data":"f11c05387ca356617149115719029b61dd04c0df70369ee3f1d83b306b49c6bb"}
Sep 09 22:07:00.336208 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.336129    2578 scope.go:117] "RemoveContainer" containerID="479413109ee04cd9fabcba227f65f4d48654bbc8802e1fa8722e7d3a43783421"
Sep 09 22:07:00.345986 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.345954    2578 scope.go:117] "RemoveContainer" containerID="479413109ee04cd9fabcba227f65f4d48654bbc8802e1fa8722e7d3a43783421"
Sep 09 22:07:00.346280 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:07:00.346263    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"479413109ee04cd9fabcba227f65f4d48654bbc8802e1fa8722e7d3a43783421\": container with ID starting with 479413109ee04cd9fabcba227f65f4d48654bbc8802e1fa8722e7d3a43783421 not found: ID does not exist" containerID="479413109ee04cd9fabcba227f65f4d48654bbc8802e1fa8722e7d3a43783421"
Sep 09 22:07:00.346343 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.346292    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"479413109ee04cd9fabcba227f65f4d48654bbc8802e1fa8722e7d3a43783421"} err="failed to get container status \"479413109ee04cd9fabcba227f65f4d48654bbc8802e1fa8722e7d3a43783421\": rpc error: code = NotFound desc = could not find container \"479413109ee04cd9fabcba227f65f4d48654bbc8802e1fa8722e7d3a43783421\": container with ID starting with 479413109ee04cd9fabcba227f65f4d48654bbc8802e1fa8722e7d3a43783421 not found: ID does not exist"
Sep 09 22:07:00.357917 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.357888    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"]
Sep 09 22:07:00.361944 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:00.361916    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-bcb351-6fd899c47-xqkx5"]
Sep 09 22:07:01.011174 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:01.011146    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8" path="/var/lib/kubelet/pods/cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8/volumes"
Sep 09 22:07:04.321931 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:04.321889    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:07:06.374363 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:06.374333    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"]
Sep 09 22:07:06.374816 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:06.374568    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr" podUID="a0cf0830-d201-4808-a84d-a026c51c36f4" containerName="istio-proxy" containerID="cri-o://1b7460d7007313951b80ab1c054d62b51da50043171279bbb6283c837fe151b2" gracePeriod=30
Sep 09 22:07:07.407390 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:07.407354    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 22:07:07.407771 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:07.407423    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr" podUID="a0cf0830-d201-4808-a84d-a026c51c36f4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 22:07:11.648928 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.648864    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:07:11.767474 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.767444    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-envoy\") pod \"a0cf0830-d201-4808-a84d-a026c51c36f4\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") "
Sep 09 22:07:11.767636 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.767504    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-data\") pod \"a0cf0830-d201-4808-a84d-a026c51c36f4\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") "
Sep 09 22:07:11.767636 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.767540    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-workload-socket\") pod \"a0cf0830-d201-4808-a84d-a026c51c36f4\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") "
Sep 09 22:07:11.767636 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.767565    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-token\") pod \"a0cf0830-d201-4808-a84d-a026c51c36f4\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") "
Sep 09 22:07:11.767636 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.767608    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7msp5\" (UniqueName: \"kubernetes.io/projected/a0cf0830-d201-4808-a84d-a026c51c36f4-kube-api-access-7msp5\") pod \"a0cf0830-d201-4808-a84d-a026c51c36f4\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") "
Sep 09 22:07:11.767863 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.767639    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-podinfo\") pod \"a0cf0830-d201-4808-a84d-a026c51c36f4\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") "
Sep 09 22:07:11.767863 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.767726    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-credential-socket\") pod \"a0cf0830-d201-4808-a84d-a026c51c36f4\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") "
Sep 09 22:07:11.767863 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.767725    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-data" (OuterVolumeSpecName: "istio-data") pod "a0cf0830-d201-4808-a84d-a026c51c36f4" (UID: "a0cf0830-d201-4808-a84d-a026c51c36f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:07:11.767863 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.767761    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0cf0830-d201-4808-a84d-a026c51c36f4-istiod-ca-cert\") pod \"a0cf0830-d201-4808-a84d-a026c51c36f4\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") "
Sep 09 22:07:11.767863 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.767787    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-workload-certs\") pod \"a0cf0830-d201-4808-a84d-a026c51c36f4\" (UID: \"a0cf0830-d201-4808-a84d-a026c51c36f4\") "
Sep 09 22:07:11.767863 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.767833    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a0cf0830-d201-4808-a84d-a026c51c36f4" (UID: "a0cf0830-d201-4808-a84d-a026c51c36f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:07:11.768178 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.768008    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a0cf0830-d201-4808-a84d-a026c51c36f4" (UID: "a0cf0830-d201-4808-a84d-a026c51c36f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:07:11.768178 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.768086    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0cf0830-d201-4808-a84d-a026c51c36f4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a0cf0830-d201-4808-a84d-a026c51c36f4" (UID: "a0cf0830-d201-4808-a84d-a026c51c36f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 22:07:11.768178 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.768165    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a0cf0830-d201-4808-a84d-a026c51c36f4" (UID: "a0cf0830-d201-4808-a84d-a026c51c36f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:07:11.768326 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.768211    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:07:11.768326 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.768233    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:07:11.768326 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.768248    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:07:11.769758 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.769731    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a0cf0830-d201-4808-a84d-a026c51c36f4" (UID: "a0cf0830-d201-4808-a84d-a026c51c36f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 22:07:11.769950 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.769929    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0cf0830-d201-4808-a84d-a026c51c36f4-kube-api-access-7msp5" (OuterVolumeSpecName: "kube-api-access-7msp5") pod "a0cf0830-d201-4808-a84d-a026c51c36f4" (UID: "a0cf0830-d201-4808-a84d-a026c51c36f4"). InnerVolumeSpecName "kube-api-access-7msp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:07:11.770084 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.770066    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a0cf0830-d201-4808-a84d-a026c51c36f4" (UID: "a0cf0830-d201-4808-a84d-a026c51c36f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:07:11.770192 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.770171    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-token" (OuterVolumeSpecName: "istio-token") pod "a0cf0830-d201-4808-a84d-a026c51c36f4" (UID: "a0cf0830-d201-4808-a84d-a026c51c36f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:07:11.869026 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.869000    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7msp5\" (UniqueName: \"kubernetes.io/projected/a0cf0830-d201-4808-a84d-a026c51c36f4-kube-api-access-7msp5\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:07:11.869026 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.869023    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:07:11.869171 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.869032    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0cf0830-d201-4808-a84d-a026c51c36f4-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:07:11.869171 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.869041    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:07:11.869171 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.869048    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:07:11.869171 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:11.869055    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0cf0830-d201-4808-a84d-a026c51c36f4-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:07:12.413924 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:12.413890    2578 generic.go:356] "Generic (PLEG): container finished" podID="a0cf0830-d201-4808-a84d-a026c51c36f4" containerID="1b7460d7007313951b80ab1c054d62b51da50043171279bbb6283c837fe151b2" exitCode=0
Sep 09 22:07:12.414118 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:12.413941    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"
Sep 09 22:07:12.414118 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:12.413962    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr" event={"ID":"a0cf0830-d201-4808-a84d-a026c51c36f4","Type":"ContainerDied","Data":"1b7460d7007313951b80ab1c054d62b51da50043171279bbb6283c837fe151b2"}
Sep 09 22:07:12.414118 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:12.414016    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr" event={"ID":"a0cf0830-d201-4808-a84d-a026c51c36f4","Type":"ContainerDied","Data":"54c0e7371a48e49a2183d38e2e87a45e0ba1efb206e3005acee3de314321ffc8"}
Sep 09 22:07:12.414118 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:12.414038    2578 scope.go:117] "RemoveContainer" containerID="1b7460d7007313951b80ab1c054d62b51da50043171279bbb6283c837fe151b2"
Sep 09 22:07:12.424597 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:12.424580    2578 scope.go:117] "RemoveContainer" containerID="5f41009adb83abb3bd6325f9133979e20db0fd44d62bfc7efef5862bab299ed7"
Sep 09 22:07:12.432902 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:12.432855    2578 scope.go:117] "RemoveContainer" containerID="1b7460d7007313951b80ab1c054d62b51da50043171279bbb6283c837fe151b2"
Sep 09 22:07:12.433411 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:07:12.433366    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b7460d7007313951b80ab1c054d62b51da50043171279bbb6283c837fe151b2\": container with ID starting with 1b7460d7007313951b80ab1c054d62b51da50043171279bbb6283c837fe151b2 not found: ID does not exist" containerID="1b7460d7007313951b80ab1c054d62b51da50043171279bbb6283c837fe151b2"
Sep 09 22:07:12.433566 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:12.433421    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b7460d7007313951b80ab1c054d62b51da50043171279bbb6283c837fe151b2"} err="failed to get container status \"1b7460d7007313951b80ab1c054d62b51da50043171279bbb6283c837fe151b2\": rpc error: code = NotFound desc = could not find container \"1b7460d7007313951b80ab1c054d62b51da50043171279bbb6283c837fe151b2\": container with ID starting with 1b7460d7007313951b80ab1c054d62b51da50043171279bbb6283c837fe151b2 not found: ID does not exist"
Sep 09 22:07:12.433566 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:12.433445    2578 scope.go:117] "RemoveContainer" containerID="5f41009adb83abb3bd6325f9133979e20db0fd44d62bfc7efef5862bab299ed7"
Sep 09 22:07:12.433823 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:07:12.433799    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f41009adb83abb3bd6325f9133979e20db0fd44d62bfc7efef5862bab299ed7\": container with ID starting with 5f41009adb83abb3bd6325f9133979e20db0fd44d62bfc7efef5862bab299ed7 not found: ID does not exist" containerID="5f41009adb83abb3bd6325f9133979e20db0fd44d62bfc7efef5862bab299ed7"
Sep 09 22:07:12.433934 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:12.433909    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f41009adb83abb3bd6325f9133979e20db0fd44d62bfc7efef5862bab299ed7"} err="failed to get container status \"5f41009adb83abb3bd6325f9133979e20db0fd44d62bfc7efef5862bab299ed7\": rpc error: code = NotFound desc = could not find container \"5f41009adb83abb3bd6325f9133979e20db0fd44d62bfc7efef5862bab299ed7\": container with ID starting with 5f41009adb83abb3bd6325f9133979e20db0fd44d62bfc7efef5862bab299ed7 not found: ID does not exist"
Sep 09 22:07:12.437007 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:12.436988    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"]
Sep 09 22:07:12.439582 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:12.439560    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-bcb351-openshift-default-6586f4bc6d-lpncr"]
Sep 09 22:07:13.003805 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:13.003772    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0cf0830-d201-4808-a84d-a026c51c36f4" path="/var/lib/kubelet/pods/a0cf0830-d201-4808-a84d-a026c51c36f4/volumes"
Sep 09 22:07:23.370476 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:23.370445    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:07:35.968033 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:07:35.967997    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:08:01.199228 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:01.199197    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 22:08:01.205663 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:01.205640    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 22:08:29.278239 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:29.278200    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"]
Sep 09 22:08:29.278799 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:29.278467    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8" podUID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerName="istio-proxy" containerID="cri-o://ff41572f0c31e947dc64f2f509cb4fd88cb134f331a857bc012a6ad2a22ff8cc" gracePeriod=30
Sep 09 22:08:29.918214 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:29.918180    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 22:08:29.918386 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:29.918245    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8" podUID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 22:08:34.533469 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.533445    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:08:34.599126 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.599060    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-envoy\") pod \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") "
Sep 09 22:08:34.599126 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.599116    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-podinfo\") pod \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") "
Sep 09 22:08:34.599330 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.599140    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-workload-socket\") pod \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") "
Sep 09 22:08:34.599330 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.599162    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-token\") pod \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") "
Sep 09 22:08:34.599330 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.599186    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-data\") pod \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") "
Sep 09 22:08:34.599330 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.599200    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istiod-ca-cert\") pod \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") "
Sep 09 22:08:34.599330 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.599240    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-credential-socket\") pod \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") "
Sep 09 22:08:34.599587 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.599483    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4397d91e-a780-4e4d-80a5-a58a2cbcef2f" (UID: "4397d91e-a780-4e4d-80a5-a58a2cbcef2f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:08:34.599640 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.599578    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4397d91e-a780-4e4d-80a5-a58a2cbcef2f" (UID: "4397d91e-a780-4e4d-80a5-a58a2cbcef2f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:08:34.599640 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.599607    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-workload-certs\") pod \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") "
Sep 09 22:08:34.599742 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.599689    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbq88\" (UniqueName: \"kubernetes.io/projected/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-kube-api-access-sbq88\") pod \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\" (UID: \"4397d91e-a780-4e4d-80a5-a58a2cbcef2f\") "
Sep 09 22:08:34.599821 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.599765    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4397d91e-a780-4e4d-80a5-a58a2cbcef2f" (UID: "4397d91e-a780-4e4d-80a5-a58a2cbcef2f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 22:08:34.599891 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.599871    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-data" (OuterVolumeSpecName: "istio-data") pod "4397d91e-a780-4e4d-80a5-a58a2cbcef2f" (UID: "4397d91e-a780-4e4d-80a5-a58a2cbcef2f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:08:34.600123 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.600101    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:34.600249 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.600231    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:34.600320 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.600247    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:34.600320 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.600263    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:34.600320 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.600158    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4397d91e-a780-4e4d-80a5-a58a2cbcef2f" (UID: "4397d91e-a780-4e4d-80a5-a58a2cbcef2f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:08:34.601806 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.601785    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4397d91e-a780-4e4d-80a5-a58a2cbcef2f" (UID: "4397d91e-a780-4e4d-80a5-a58a2cbcef2f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:08:34.601806 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.601787    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4397d91e-a780-4e4d-80a5-a58a2cbcef2f" (UID: "4397d91e-a780-4e4d-80a5-a58a2cbcef2f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 22:08:34.602055 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.602032    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-token" (OuterVolumeSpecName: "istio-token") pod "4397d91e-a780-4e4d-80a5-a58a2cbcef2f" (UID: "4397d91e-a780-4e4d-80a5-a58a2cbcef2f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:08:34.602136 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.602118    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-kube-api-access-sbq88" (OuterVolumeSpecName: "kube-api-access-sbq88") pod "4397d91e-a780-4e4d-80a5-a58a2cbcef2f" (UID: "4397d91e-a780-4e4d-80a5-a58a2cbcef2f"). InnerVolumeSpecName "kube-api-access-sbq88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:08:34.701415 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.701391    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:34.701415 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.701413    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:34.701555 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.701422    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbq88\" (UniqueName: \"kubernetes.io/projected/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-kube-api-access-sbq88\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:34.701555 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.701431    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:34.701555 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.701439    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4397d91e-a780-4e4d-80a5-a58a2cbcef2f-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:34.901442 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.901379    2578 generic.go:356] "Generic (PLEG): container finished" podID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerID="ff41572f0c31e947dc64f2f509cb4fd88cb134f331a857bc012a6ad2a22ff8cc" exitCode=0
Sep 09 22:08:34.901442 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.901436    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"
Sep 09 22:08:34.901619 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.901473    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8" event={"ID":"4397d91e-a780-4e4d-80a5-a58a2cbcef2f","Type":"ContainerDied","Data":"ff41572f0c31e947dc64f2f509cb4fd88cb134f331a857bc012a6ad2a22ff8cc"}
Sep 09 22:08:34.901619 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.901562    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8" event={"ID":"4397d91e-a780-4e4d-80a5-a58a2cbcef2f","Type":"ContainerDied","Data":"5b34c7775c64b7f6dc98ea55b0a19e1cb4b875d30f96639b888e9bec44cb42bb"}
Sep 09 22:08:34.901619 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.901577    2578 scope.go:117] "RemoveContainer" containerID="ff41572f0c31e947dc64f2f509cb4fd88cb134f331a857bc012a6ad2a22ff8cc"
Sep 09 22:08:34.911301 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.911278    2578 scope.go:117] "RemoveContainer" containerID="4e6d5494a6d871ec5fea58eeee9459608e05dd5746c43878b2c209e727ca794a"
Sep 09 22:08:34.919392 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.919372    2578 scope.go:117] "RemoveContainer" containerID="ff41572f0c31e947dc64f2f509cb4fd88cb134f331a857bc012a6ad2a22ff8cc"
Sep 09 22:08:34.919738 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:08:34.919710    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff41572f0c31e947dc64f2f509cb4fd88cb134f331a857bc012a6ad2a22ff8cc\": container with ID starting with ff41572f0c31e947dc64f2f509cb4fd88cb134f331a857bc012a6ad2a22ff8cc not found: ID does not exist" containerID="ff41572f0c31e947dc64f2f509cb4fd88cb134f331a857bc012a6ad2a22ff8cc"
Sep 09 22:08:34.919850 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.919752    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff41572f0c31e947dc64f2f509cb4fd88cb134f331a857bc012a6ad2a22ff8cc"} err="failed to get container status \"ff41572f0c31e947dc64f2f509cb4fd88cb134f331a857bc012a6ad2a22ff8cc\": rpc error: code = NotFound desc = could not find container \"ff41572f0c31e947dc64f2f509cb4fd88cb134f331a857bc012a6ad2a22ff8cc\": container with ID starting with ff41572f0c31e947dc64f2f509cb4fd88cb134f331a857bc012a6ad2a22ff8cc not found: ID does not exist"
Sep 09 22:08:34.919850 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.919774    2578 scope.go:117] "RemoveContainer" containerID="4e6d5494a6d871ec5fea58eeee9459608e05dd5746c43878b2c209e727ca794a"
Sep 09 22:08:34.920173 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:08:34.920142    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e6d5494a6d871ec5fea58eeee9459608e05dd5746c43878b2c209e727ca794a\": container with ID starting with 4e6d5494a6d871ec5fea58eeee9459608e05dd5746c43878b2c209e727ca794a not found: ID does not exist" containerID="4e6d5494a6d871ec5fea58eeee9459608e05dd5746c43878b2c209e727ca794a"
Sep 09 22:08:34.920260 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.920177    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e6d5494a6d871ec5fea58eeee9459608e05dd5746c43878b2c209e727ca794a"} err="failed to get container status \"4e6d5494a6d871ec5fea58eeee9459608e05dd5746c43878b2c209e727ca794a\": rpc error: code = NotFound desc = could not find container \"4e6d5494a6d871ec5fea58eeee9459608e05dd5746c43878b2c209e727ca794a\": container with ID starting with 4e6d5494a6d871ec5fea58eeee9459608e05dd5746c43878b2c209e727ca794a not found: ID does not exist"
Sep 09 22:08:34.922643 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.922620    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"]
Sep 09 22:08:34.928483 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:34.928464    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-100032-openshift-default-5b4c44ffd-tp4d8"]
Sep 09 22:08:35.003349 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:35.003319    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" path="/var/lib/kubelet/pods/4397d91e-a780-4e4d-80a5-a58a2cbcef2f/volumes"
Sep 09 22:08:43.377764 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:43.377724    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:08:47.478237 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:47.478199    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"]
Sep 09 22:08:47.478711 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:47.478451    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5" podUID="5ba01acc-614c-48e1-b608-9adbef027488" containerName="istio-proxy" containerID="cri-o://d9cf05e64f2d72e4e938550657252a8da91168f7f6fce5b25a33be58591b5817" gracePeriod=30
Sep 09 22:08:47.713125 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:47.713085    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 22:08:47.713295 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:47.713158    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5" podUID="5ba01acc-614c-48e1-b608-9adbef027488" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 22:08:52.730208 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.730184    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:08:52.850216 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.850188    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ba01acc-614c-48e1-b608-9adbef027488-istio-podinfo\") pod \"5ba01acc-614c-48e1-b608-9adbef027488\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") "
Sep 09 22:08:52.850386 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.850260    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ba01acc-614c-48e1-b608-9adbef027488-istiod-ca-cert\") pod \"5ba01acc-614c-48e1-b608-9adbef027488\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") "
Sep 09 22:08:52.850386 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.850286    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-istio-envoy\") pod \"5ba01acc-614c-48e1-b608-9adbef027488\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") "
Sep 09 22:08:52.850386 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.850315    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-istio-data\") pod \"5ba01acc-614c-48e1-b608-9adbef027488\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") "
Sep 09 22:08:52.850386 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.850333    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-workload-socket\") pod \"5ba01acc-614c-48e1-b608-9adbef027488\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") "
Sep 09 22:08:52.850386 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.850352    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ba01acc-614c-48e1-b608-9adbef027488-istio-token\") pod \"5ba01acc-614c-48e1-b608-9adbef027488\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") "
Sep 09 22:08:52.850662 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.850386    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99q2q\" (UniqueName: \"kubernetes.io/projected/5ba01acc-614c-48e1-b608-9adbef027488-kube-api-access-99q2q\") pod \"5ba01acc-614c-48e1-b608-9adbef027488\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") "
Sep 09 22:08:52.850662 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.850565    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-workload-certs\") pod \"5ba01acc-614c-48e1-b608-9adbef027488\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") "
Sep 09 22:08:52.850662 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.850610    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-credential-socket\") pod \"5ba01acc-614c-48e1-b608-9adbef027488\" (UID: \"5ba01acc-614c-48e1-b608-9adbef027488\") "
Sep 09 22:08:52.850826 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.850688    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-istio-data" (OuterVolumeSpecName: "istio-data") pod "5ba01acc-614c-48e1-b608-9adbef027488" (UID: "5ba01acc-614c-48e1-b608-9adbef027488"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:08:52.850826 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.850702    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5ba01acc-614c-48e1-b608-9adbef027488" (UID: "5ba01acc-614c-48e1-b608-9adbef027488"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:08:52.850826 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.850771    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ba01acc-614c-48e1-b608-9adbef027488-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5ba01acc-614c-48e1-b608-9adbef027488" (UID: "5ba01acc-614c-48e1-b608-9adbef027488"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 22:08:52.851011 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.850919    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5ba01acc-614c-48e1-b608-9adbef027488" (UID: "5ba01acc-614c-48e1-b608-9adbef027488"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:08:52.851011 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.850937    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5ba01acc-614c-48e1-b608-9adbef027488" (UID: "5ba01acc-614c-48e1-b608-9adbef027488"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:08:52.851133 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.851115    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ba01acc-614c-48e1-b608-9adbef027488-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:52.851198 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.851139    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:52.851198 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.851152    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:52.851198 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.851166    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:52.851198 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.851180    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:52.852441 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.852416    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5ba01acc-614c-48e1-b608-9adbef027488-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5ba01acc-614c-48e1-b608-9adbef027488" (UID: "5ba01acc-614c-48e1-b608-9adbef027488"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 22:08:52.852551 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.852532    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5ba01acc-614c-48e1-b608-9adbef027488" (UID: "5ba01acc-614c-48e1-b608-9adbef027488"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:08:52.852871 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.852844    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ba01acc-614c-48e1-b608-9adbef027488-istio-token" (OuterVolumeSpecName: "istio-token") pod "5ba01acc-614c-48e1-b608-9adbef027488" (UID: "5ba01acc-614c-48e1-b608-9adbef027488"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:08:52.852871 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.852855    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ba01acc-614c-48e1-b608-9adbef027488-kube-api-access-99q2q" (OuterVolumeSpecName: "kube-api-access-99q2q") pod "5ba01acc-614c-48e1-b608-9adbef027488" (UID: "5ba01acc-614c-48e1-b608-9adbef027488"). InnerVolumeSpecName "kube-api-access-99q2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:08:52.952202 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.952175    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ba01acc-614c-48e1-b608-9adbef027488-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:52.952202 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.952198    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ba01acc-614c-48e1-b608-9adbef027488-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:52.952360 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.952207    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99q2q\" (UniqueName: \"kubernetes.io/projected/5ba01acc-614c-48e1-b608-9adbef027488-kube-api-access-99q2q\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:52.952360 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:52.952216    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ba01acc-614c-48e1-b608-9adbef027488-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:53.011874 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:53.011851    2578 generic.go:356] "Generic (PLEG): container finished" podID="5ba01acc-614c-48e1-b608-9adbef027488" containerID="d9cf05e64f2d72e4e938550657252a8da91168f7f6fce5b25a33be58591b5817" exitCode=0
Sep 09 22:08:53.011993 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:53.011884    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"
Sep 09 22:08:53.011993 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:53.011893    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5" event={"ID":"5ba01acc-614c-48e1-b608-9adbef027488","Type":"ContainerDied","Data":"d9cf05e64f2d72e4e938550657252a8da91168f7f6fce5b25a33be58591b5817"}
Sep 09 22:08:53.011993 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:53.011933    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5" event={"ID":"5ba01acc-614c-48e1-b608-9adbef027488","Type":"ContainerDied","Data":"534343a2d54cf986cf72048ddeb9e3aabdf8209f85c08501f29e557afe25aca6"}
Sep 09 22:08:53.011993 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:53.011956    2578 scope.go:117] "RemoveContainer" containerID="d9cf05e64f2d72e4e938550657252a8da91168f7f6fce5b25a33be58591b5817"
Sep 09 22:08:53.021113 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:53.021096    2578 scope.go:117] "RemoveContainer" containerID="1f6637178dcd4cbfaa3a2c70bf2451e31a1ee9a38ce72f864cc5dcd96f299a62"
Sep 09 22:08:53.028952 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:53.028919    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"]
Sep 09 22:08:53.029834 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:53.029818    2578 scope.go:117] "RemoveContainer" containerID="d9cf05e64f2d72e4e938550657252a8da91168f7f6fce5b25a33be58591b5817"
Sep 09 22:08:53.030124 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:08:53.030101    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9cf05e64f2d72e4e938550657252a8da91168f7f6fce5b25a33be58591b5817\": container with ID starting with d9cf05e64f2d72e4e938550657252a8da91168f7f6fce5b25a33be58591b5817 not found: ID does not exist" containerID="d9cf05e64f2d72e4e938550657252a8da91168f7f6fce5b25a33be58591b5817"
Sep 09 22:08:53.030253 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:53.030131    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9cf05e64f2d72e4e938550657252a8da91168f7f6fce5b25a33be58591b5817"} err="failed to get container status \"d9cf05e64f2d72e4e938550657252a8da91168f7f6fce5b25a33be58591b5817\": rpc error: code = NotFound desc = could not find container \"d9cf05e64f2d72e4e938550657252a8da91168f7f6fce5b25a33be58591b5817\": container with ID starting with d9cf05e64f2d72e4e938550657252a8da91168f7f6fce5b25a33be58591b5817 not found: ID does not exist"
Sep 09 22:08:53.030253 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:53.030154    2578 scope.go:117] "RemoveContainer" containerID="1f6637178dcd4cbfaa3a2c70bf2451e31a1ee9a38ce72f864cc5dcd96f299a62"
Sep 09 22:08:53.030456 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:08:53.030435    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f6637178dcd4cbfaa3a2c70bf2451e31a1ee9a38ce72f864cc5dcd96f299a62\": container with ID starting with 1f6637178dcd4cbfaa3a2c70bf2451e31a1ee9a38ce72f864cc5dcd96f299a62 not found: ID does not exist" containerID="1f6637178dcd4cbfaa3a2c70bf2451e31a1ee9a38ce72f864cc5dcd96f299a62"
Sep 09 22:08:53.030543 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:53.030461    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f6637178dcd4cbfaa3a2c70bf2451e31a1ee9a38ce72f864cc5dcd96f299a62"} err="failed to get container status \"1f6637178dcd4cbfaa3a2c70bf2451e31a1ee9a38ce72f864cc5dcd96f299a62\": rpc error: code = NotFound desc = could not find container \"1f6637178dcd4cbfaa3a2c70bf2451e31a1ee9a38ce72f864cc5dcd96f299a62\": container with ID starting with 1f6637178dcd4cbfaa3a2c70bf2451e31a1ee9a38ce72f864cc5dcd96f299a62 not found: ID does not exist"
Sep 09 22:08:53.033902 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:53.033883    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-855e86-openshift-default-787f847fc8-vggg5"]
Sep 09 22:08:55.002818 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:55.002743    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ba01acc-614c-48e1-b608-9adbef027488" path="/var/lib/kubelet/pods/5ba01acc-614c-48e1-b608-9adbef027488/volumes"
Sep 09 22:08:58.350648 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.350613    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"]
Sep 09 22:08:58.351221 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.350838    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289" podUID="ce598ded-25f8-44d8-8285-e8c0122f1e21" containerName="maas-api" containerID="cri-o://29acde326e180152d6ff92a012f1592551a9c91240487c9e5129c7c106e3a179" gracePeriod=30
Sep 09 22:08:58.386292 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.386257    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2d3bc8-29816520-lmb48"]
Sep 09 22:08:58.390063 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.390036    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-2d3bc8-29816520-lmb48"]
Sep 09 22:08:58.601500 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.601437    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 22:08:58.695542 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.695511    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnjl5\" (UniqueName: \"kubernetes.io/projected/ce598ded-25f8-44d8-8285-e8c0122f1e21-kube-api-access-lnjl5\") pod \"ce598ded-25f8-44d8-8285-e8c0122f1e21\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") "
Sep 09 22:08:58.695686 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.695610    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce598ded-25f8-44d8-8285-e8c0122f1e21-metrics-certs\") pod \"ce598ded-25f8-44d8-8285-e8c0122f1e21\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") "
Sep 09 22:08:58.695686 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.695669    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce598ded-25f8-44d8-8285-e8c0122f1e21-openshift-service-ca\") pod \"ce598ded-25f8-44d8-8285-e8c0122f1e21\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") "
Sep 09 22:08:58.695777 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.695713    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce598ded-25f8-44d8-8285-e8c0122f1e21-maas-api-tls\") pod \"ce598ded-25f8-44d8-8285-e8c0122f1e21\" (UID: \"ce598ded-25f8-44d8-8285-e8c0122f1e21\") "
Sep 09 22:08:58.696075 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.696044    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce598ded-25f8-44d8-8285-e8c0122f1e21-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ce598ded-25f8-44d8-8285-e8c0122f1e21" (UID: "ce598ded-25f8-44d8-8285-e8c0122f1e21"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 22:08:58.697712 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.697679    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce598ded-25f8-44d8-8285-e8c0122f1e21-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ce598ded-25f8-44d8-8285-e8c0122f1e21" (UID: "ce598ded-25f8-44d8-8285-e8c0122f1e21"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 22:08:58.697800 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.697724    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce598ded-25f8-44d8-8285-e8c0122f1e21-kube-api-access-lnjl5" (OuterVolumeSpecName: "kube-api-access-lnjl5") pod "ce598ded-25f8-44d8-8285-e8c0122f1e21" (UID: "ce598ded-25f8-44d8-8285-e8c0122f1e21"). InnerVolumeSpecName "kube-api-access-lnjl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:08:58.697800 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.697727    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce598ded-25f8-44d8-8285-e8c0122f1e21-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ce598ded-25f8-44d8-8285-e8c0122f1e21" (UID: "ce598ded-25f8-44d8-8285-e8c0122f1e21"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 22:08:58.797317 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.797285    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce598ded-25f8-44d8-8285-e8c0122f1e21-metrics-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:58.797317 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.797311    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce598ded-25f8-44d8-8285-e8c0122f1e21-openshift-service-ca\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:58.797317 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.797322    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce598ded-25f8-44d8-8285-e8c0122f1e21-maas-api-tls\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:58.797554 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:58.797330    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnjl5\" (UniqueName: \"kubernetes.io/projected/ce598ded-25f8-44d8-8285-e8c0122f1e21-kube-api-access-lnjl5\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:08:59.003078 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:59.003002    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a1328b1-f64f-4f75-af79-0e6e07926705" path="/var/lib/kubelet/pods/8a1328b1-f64f-4f75-af79-0e6e07926705/volumes"
Sep 09 22:08:59.046699 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:59.046670    2578 generic.go:356] "Generic (PLEG): container finished" podID="ce598ded-25f8-44d8-8285-e8c0122f1e21" containerID="29acde326e180152d6ff92a012f1592551a9c91240487c9e5129c7c106e3a179" exitCode=0
Sep 09 22:08:59.046823 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:59.046712    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"
Sep 09 22:08:59.046823 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:59.046739    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289" event={"ID":"ce598ded-25f8-44d8-8285-e8c0122f1e21","Type":"ContainerDied","Data":"29acde326e180152d6ff92a012f1592551a9c91240487c9e5129c7c106e3a179"}
Sep 09 22:08:59.046823 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:59.046777    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289" event={"ID":"ce598ded-25f8-44d8-8285-e8c0122f1e21","Type":"ContainerDied","Data":"05a102e40d28512e931e76467da492dde1bdd6fead9f4773126a0ebd9dc46816"}
Sep 09 22:08:59.046823 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:59.046797    2578 scope.go:117] "RemoveContainer" containerID="29acde326e180152d6ff92a012f1592551a9c91240487c9e5129c7c106e3a179"
Sep 09 22:08:59.055842 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:59.055824    2578 scope.go:117] "RemoveContainer" containerID="29acde326e180152d6ff92a012f1592551a9c91240487c9e5129c7c106e3a179"
Sep 09 22:08:59.056157 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:08:59.056132    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29acde326e180152d6ff92a012f1592551a9c91240487c9e5129c7c106e3a179\": container with ID starting with 29acde326e180152d6ff92a012f1592551a9c91240487c9e5129c7c106e3a179 not found: ID does not exist" containerID="29acde326e180152d6ff92a012f1592551a9c91240487c9e5129c7c106e3a179"
Sep 09 22:08:59.056230 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:59.056164    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29acde326e180152d6ff92a012f1592551a9c91240487c9e5129c7c106e3a179"} err="failed to get container status \"29acde326e180152d6ff92a012f1592551a9c91240487c9e5129c7c106e3a179\": rpc error: code = NotFound desc = could not find container \"29acde326e180152d6ff92a012f1592551a9c91240487c9e5129c7c106e3a179\": container with ID starting with 29acde326e180152d6ff92a012f1592551a9c91240487c9e5129c7c106e3a179 not found: ID does not exist"
Sep 09 22:08:59.064385 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:59.064363    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"]
Sep 09 22:08:59.068594 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:08:59.068573    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2d3bc8-69dc69dc59-5l289"]
Sep 09 22:09:01.003033 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:01.003001    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce598ded-25f8-44d8-8285-e8c0122f1e21" path="/var/lib/kubelet/pods/ce598ded-25f8-44d8-8285-e8c0122f1e21/volumes"
Sep 09 22:09:01.365930 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:01.365899    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:09:01.507482 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:01.507448    2578 scope.go:117] "RemoveContainer" containerID="c3891adb328764ed04ddaf23a25f55918d925f615c0b94b1a529aeacbb83ed61"
Sep 09 22:09:05.673244 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:05.673212    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"]
Sep 09 22:09:05.673690 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:05.673458    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r" podUID="6d131295-a2b3-4629-a34e-172b8e8bb828" containerName="istio-proxy" containerID="cri-o://7b6c5aff354741df7bec578f53c97e138826adae81884b4eb8d16e15a4255bca" gracePeriod=30
Sep 09 22:09:06.064905 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.064871    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv"]
Sep 09 22:09:06.065290 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065275    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0cf0830-d201-4808-a84d-a026c51c36f4" containerName="istio-proxy"
Sep 09 22:09:06.065290 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065292    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0cf0830-d201-4808-a84d-a026c51c36f4" containerName="istio-proxy"
Sep 09 22:09:06.065392 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065316    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ba01acc-614c-48e1-b608-9adbef027488" containerName="istio-proxy"
Sep 09 22:09:06.065392 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065322    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ba01acc-614c-48e1-b608-9adbef027488" containerName="istio-proxy"
Sep 09 22:09:06.065392 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065328    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0cf0830-d201-4808-a84d-a026c51c36f4" containerName="istio-proxy"
Sep 09 22:09:06.065392 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065334    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0cf0830-d201-4808-a84d-a026c51c36f4" containerName="istio-proxy"
Sep 09 22:09:06.065392 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065342    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8" containerName="maas-api"
Sep 09 22:09:06.065392 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065347    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8" containerName="maas-api"
Sep 09 22:09:06.065392 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065357    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerName="istio-proxy"
Sep 09 22:09:06.065392 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065362    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerName="istio-proxy"
Sep 09 22:09:06.065392 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065370    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerName="istio-proxy"
Sep 09 22:09:06.065392 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065375    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerName="istio-proxy"
Sep 09 22:09:06.065392 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065380    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce598ded-25f8-44d8-8285-e8c0122f1e21" containerName="maas-api"
Sep 09 22:09:06.065392 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065385    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce598ded-25f8-44d8-8285-e8c0122f1e21" containerName="maas-api"
Sep 09 22:09:06.065392 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065393    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ba01acc-614c-48e1-b608-9adbef027488" containerName="istio-proxy"
Sep 09 22:09:06.065392 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065399    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ba01acc-614c-48e1-b608-9adbef027488" containerName="istio-proxy"
Sep 09 22:09:06.065852 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065457    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ba01acc-614c-48e1-b608-9adbef027488" containerName="istio-proxy"
Sep 09 22:09:06.065852 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065465    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ba01acc-614c-48e1-b608-9adbef027488" containerName="istio-proxy"
Sep 09 22:09:06.065852 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065472    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerName="istio-proxy"
Sep 09 22:09:06.065852 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065477    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0cf0830-d201-4808-a84d-a026c51c36f4" containerName="istio-proxy"
Sep 09 22:09:06.065852 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065483    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0cf0830-d201-4808-a84d-a026c51c36f4" containerName="istio-proxy"
Sep 09 22:09:06.065852 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065489    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4397d91e-a780-4e4d-80a5-a58a2cbcef2f" containerName="istio-proxy"
Sep 09 22:09:06.065852 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065495    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc42ee15-ce0d-45a0-b6da-4f3a7b5d78e8" containerName="maas-api"
Sep 09 22:09:06.065852 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.065502    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce598ded-25f8-44d8-8285-e8c0122f1e21" containerName="maas-api"
Sep 09 22:09:06.068666 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.068649    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv"
Sep 09 22:09:06.071299 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.071279    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 22:09:06.071953 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.071935    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65jqq\""
Sep 09 22:09:06.078907 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.078884    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv"]
Sep 09 22:09:06.163111 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.163079    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3184537a-9bb6-4dff-ae62-6e22dec826c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv\" (UID: \"3184537a-9bb6-4dff-ae62-6e22dec826c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv"
Sep 09 22:09:06.263487 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.263464    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3184537a-9bb6-4dff-ae62-6e22dec826c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv\" (UID: \"3184537a-9bb6-4dff-ae62-6e22dec826c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv"
Sep 09 22:09:06.264073 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.264053    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3184537a-9bb6-4dff-ae62-6e22dec826c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv\" (UID: \"3184537a-9bb6-4dff-ae62-6e22dec826c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv"
Sep 09 22:09:06.379836 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.379764    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv"
Sep 09 22:09:06.658855 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:06.658827    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv"]
Sep 09 22:09:06.660413 ip-10-0-140-157 kubenswrapper[2578]: W0909 22:09:06.660386    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3184537a_9bb6_4dff_ae62_6e22dec826c3.slice/crio-842159987b712668f7620002e965472432462dbe0c7ab149f0b062e38f7ecfed WatchSource:0}: Error finding container 842159987b712668f7620002e965472432462dbe0c7ab149f0b062e38f7ecfed: Status 404 returned error can't find the container with id 842159987b712668f7620002e965472432462dbe0c7ab149f0b062e38f7ecfed
Sep 09 22:09:07.092686 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:07.092659    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv" event={"ID":"3184537a-9bb6-4dff-ae62-6e22dec826c3","Type":"ContainerStarted","Data":"6f245e66ed7712e75a50b34b5f7c6bf9004a949290e2834f3bd3af4c4567865f"}
Sep 09 22:09:07.093027 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:07.092702    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv" event={"ID":"3184537a-9bb6-4dff-ae62-6e22dec826c3","Type":"ContainerStarted","Data":"842159987b712668f7620002e965472432462dbe0c7ab149f0b062e38f7ecfed"}
Sep 09 22:09:07.109704 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:07.109661    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv" podStartSLOduration=1.109647561 podStartE2EDuration="1.109647561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:09:07.106790077 +0000 UTC m=+2166.579754987" watchObservedRunningTime="2026-09-09 22:09:07.109647561 +0000 UTC m=+2166.582612471"
Sep 09 22:09:08.099669 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:08.099632    2578 generic.go:356] "Generic (PLEG): container finished" podID="3184537a-9bb6-4dff-ae62-6e22dec826c3" containerID="6f245e66ed7712e75a50b34b5f7c6bf9004a949290e2834f3bd3af4c4567865f" exitCode=0
Sep 09 22:09:08.100060 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:08.099671    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv" event={"ID":"3184537a-9bb6-4dff-ae62-6e22dec826c3","Type":"ContainerDied","Data":"6f245e66ed7712e75a50b34b5f7c6bf9004a949290e2834f3bd3af4c4567865f"}
Sep 09 22:09:09.233138 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:09.233116    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv"
Sep 09 22:09:09.390448 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:09.390378    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3184537a-9bb6-4dff-ae62-6e22dec826c3-maas-api-service-ca\") pod \"3184537a-9bb6-4dff-ae62-6e22dec826c3\" (UID: \"3184537a-9bb6-4dff-ae62-6e22dec826c3\") "
Sep 09 22:09:09.390743 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:09.390719    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3184537a-9bb6-4dff-ae62-6e22dec826c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3184537a-9bb6-4dff-ae62-6e22dec826c3" (UID: "3184537a-9bb6-4dff-ae62-6e22dec826c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 22:09:09.491523 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:09.491495    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3184537a-9bb6-4dff-ae62-6e22dec826c3-maas-api-service-ca\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:10.111955 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:10.111922    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv"
Sep 09 22:09:10.112149 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:10.111927    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv" event={"ID":"3184537a-9bb6-4dff-ae62-6e22dec826c3","Type":"ContainerDied","Data":"842159987b712668f7620002e965472432462dbe0c7ab149f0b062e38f7ecfed"}
Sep 09 22:09:10.112149 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:10.112059    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="842159987b712668f7620002e965472432462dbe0c7ab149f0b062e38f7ecfed"
Sep 09 22:09:10.918933 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:10.918903    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 22:09:11.106049 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106023    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-data\") pod \"6d131295-a2b3-4629-a34e-172b8e8bb828\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") "
Sep 09 22:09:11.106196 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106084    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vl8v\" (UniqueName: \"kubernetes.io/projected/6d131295-a2b3-4629-a34e-172b8e8bb828-kube-api-access-9vl8v\") pod \"6d131295-a2b3-4629-a34e-172b8e8bb828\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") "
Sep 09 22:09:11.106196 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106107    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-workload-socket\") pod \"6d131295-a2b3-4629-a34e-172b8e8bb828\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") "
Sep 09 22:09:11.106196 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106141    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-podinfo\") pod \"6d131295-a2b3-4629-a34e-172b8e8bb828\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") "
Sep 09 22:09:11.106196 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106167    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-envoy\") pod \"6d131295-a2b3-4629-a34e-172b8e8bb828\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") "
Sep 09 22:09:11.106196 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106193    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-credential-socket\") pod \"6d131295-a2b3-4629-a34e-172b8e8bb828\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") "
Sep 09 22:09:11.106452 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106220    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-token\") pod \"6d131295-a2b3-4629-a34e-172b8e8bb828\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") "
Sep 09 22:09:11.106452 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106245    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-workload-certs\") pod \"6d131295-a2b3-4629-a34e-172b8e8bb828\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") "
Sep 09 22:09:11.106452 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106274    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-data" (OuterVolumeSpecName: "istio-data") pod "6d131295-a2b3-4629-a34e-172b8e8bb828" (UID: "6d131295-a2b3-4629-a34e-172b8e8bb828"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:09:11.106452 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106362    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d131295-a2b3-4629-a34e-172b8e8bb828-istiod-ca-cert\") pod \"6d131295-a2b3-4629-a34e-172b8e8bb828\" (UID: \"6d131295-a2b3-4629-a34e-172b8e8bb828\") "
Sep 09 22:09:11.106675 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106457    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6d131295-a2b3-4629-a34e-172b8e8bb828" (UID: "6d131295-a2b3-4629-a34e-172b8e8bb828"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:09:11.106675 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106469    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6d131295-a2b3-4629-a34e-172b8e8bb828" (UID: "6d131295-a2b3-4629-a34e-172b8e8bb828"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:09:11.106675 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106606    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6d131295-a2b3-4629-a34e-172b8e8bb828" (UID: "6d131295-a2b3-4629-a34e-172b8e8bb828"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:09:11.106847 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106752    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:11.106847 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106756    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d131295-a2b3-4629-a34e-172b8e8bb828-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6d131295-a2b3-4629-a34e-172b8e8bb828" (UID: "6d131295-a2b3-4629-a34e-172b8e8bb828"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 22:09:11.106847 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106770    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:11.106847 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106785    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:11.106847 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.106798    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:11.108460 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.108429    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-token" (OuterVolumeSpecName: "istio-token") pod "6d131295-a2b3-4629-a34e-172b8e8bb828" (UID: "6d131295-a2b3-4629-a34e-172b8e8bb828"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:09:11.108460 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.108447    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6d131295-a2b3-4629-a34e-172b8e8bb828" (UID: "6d131295-a2b3-4629-a34e-172b8e8bb828"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:09:11.108598 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.108533    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6d131295-a2b3-4629-a34e-172b8e8bb828" (UID: "6d131295-a2b3-4629-a34e-172b8e8bb828"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 22:09:11.108598 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.108540    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d131295-a2b3-4629-a34e-172b8e8bb828-kube-api-access-9vl8v" (OuterVolumeSpecName: "kube-api-access-9vl8v") pod "6d131295-a2b3-4629-a34e-172b8e8bb828" (UID: "6d131295-a2b3-4629-a34e-172b8e8bb828"). InnerVolumeSpecName "kube-api-access-9vl8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:09:11.120405 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.120383    2578 generic.go:356] "Generic (PLEG): container finished" podID="6d131295-a2b3-4629-a34e-172b8e8bb828" containerID="7b6c5aff354741df7bec578f53c97e138826adae81884b4eb8d16e15a4255bca" exitCode=0
Sep 09 22:09:11.120531 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.120431    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"
Sep 09 22:09:11.120531 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.120465    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r" event={"ID":"6d131295-a2b3-4629-a34e-172b8e8bb828","Type":"ContainerDied","Data":"7b6c5aff354741df7bec578f53c97e138826adae81884b4eb8d16e15a4255bca"}
Sep 09 22:09:11.120618 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.120531    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r" event={"ID":"6d131295-a2b3-4629-a34e-172b8e8bb828","Type":"ContainerDied","Data":"41401836dc3f26d48bd72e35c5b6299a2edea733247911a83932da55af9e5a65"}
Sep 09 22:09:11.120618 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.120560    2578 scope.go:117] "RemoveContainer" containerID="7b6c5aff354741df7bec578f53c97e138826adae81884b4eb8d16e15a4255bca"
Sep 09 22:09:11.130267 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.130252    2578 scope.go:117] "RemoveContainer" containerID="204c082d1ac5a269f97831cd1d00162077b7e08b48045d566b7ccdea6005f6d3"
Sep 09 22:09:11.138420 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.138402    2578 scope.go:117] "RemoveContainer" containerID="7b6c5aff354741df7bec578f53c97e138826adae81884b4eb8d16e15a4255bca"
Sep 09 22:09:11.138666 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:09:11.138643    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b6c5aff354741df7bec578f53c97e138826adae81884b4eb8d16e15a4255bca\": container with ID starting with 7b6c5aff354741df7bec578f53c97e138826adae81884b4eb8d16e15a4255bca not found: ID does not exist" containerID="7b6c5aff354741df7bec578f53c97e138826adae81884b4eb8d16e15a4255bca"
Sep 09 22:09:11.138712 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.138673    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b6c5aff354741df7bec578f53c97e138826adae81884b4eb8d16e15a4255bca"} err="failed to get container status \"7b6c5aff354741df7bec578f53c97e138826adae81884b4eb8d16e15a4255bca\": rpc error: code = NotFound desc = could not find container \"7b6c5aff354741df7bec578f53c97e138826adae81884b4eb8d16e15a4255bca\": container with ID starting with 7b6c5aff354741df7bec578f53c97e138826adae81884b4eb8d16e15a4255bca not found: ID does not exist"
Sep 09 22:09:11.138712 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.138691    2578 scope.go:117] "RemoveContainer" containerID="204c082d1ac5a269f97831cd1d00162077b7e08b48045d566b7ccdea6005f6d3"
Sep 09 22:09:11.138916 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:09:11.138899    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"204c082d1ac5a269f97831cd1d00162077b7e08b48045d566b7ccdea6005f6d3\": container with ID starting with 204c082d1ac5a269f97831cd1d00162077b7e08b48045d566b7ccdea6005f6d3 not found: ID does not exist" containerID="204c082d1ac5a269f97831cd1d00162077b7e08b48045d566b7ccdea6005f6d3"
Sep 09 22:09:11.138995 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.138923    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"204c082d1ac5a269f97831cd1d00162077b7e08b48045d566b7ccdea6005f6d3"} err="failed to get container status \"204c082d1ac5a269f97831cd1d00162077b7e08b48045d566b7ccdea6005f6d3\": rpc error: code = NotFound desc = could not find container \"204c082d1ac5a269f97831cd1d00162077b7e08b48045d566b7ccdea6005f6d3\": container with ID starting with 204c082d1ac5a269f97831cd1d00162077b7e08b48045d566b7ccdea6005f6d3 not found: ID does not exist"
Sep 09 22:09:11.142776 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.142751    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"]
Sep 09 22:09:11.147426 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.147407    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-2d3bc8-openshift-default-7744dd8884-zlh6r"]
Sep 09 22:09:11.207350 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.207329    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d131295-a2b3-4629-a34e-172b8e8bb828-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:11.207350 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.207347    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vl8v\" (UniqueName: \"kubernetes.io/projected/6d131295-a2b3-4629-a34e-172b8e8bb828-kube-api-access-9vl8v\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:11.207478 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.207356    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:11.207478 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.207365    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:11.207478 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:11.207375    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d131295-a2b3-4629-a34e-172b8e8bb828-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:13.003241 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:13.003206    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d131295-a2b3-4629-a34e-172b8e8bb828" path="/var/lib/kubelet/pods/6d131295-a2b3-4629-a34e-172b8e8bb828/volumes"
Sep 09 22:09:16.850070 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:16.850034    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv"]
Sep 09 22:09:16.854667 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:16.854639    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-64fec6-tj5lv"]
Sep 09 22:09:17.004105 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:17.004067    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3184537a-9bb6-4dff-ae62-6e22dec826c3" path="/var/lib/kubelet/pods/3184537a-9bb6-4dff-ae62-6e22dec826c3/volumes"
Sep 09 22:09:19.076119 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:19.076087    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:09:23.583656 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:23.583623    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"]
Sep 09 22:09:23.584168 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:23.583830    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2" podUID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" containerName="istio-proxy" containerID="cri-o://6e16bdcb5f057677bb55ead46c65a3b0197746917be1e79fac3a6f956bdc783f" gracePeriod=30
Sep 09 22:09:28.278789 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.278750    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:09:28.289094 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.289066    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 22:09:28.289220 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.289130    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2" podUID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 22:09:28.834354 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.834330    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 22:09:28.951302 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.951211    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn9jw\" (UniqueName: \"kubernetes.io/projected/c5ac6049-fc4d-420d-b1f4-cb5788d49023-kube-api-access-bn9jw\") pod \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") "
Sep 09 22:09:28.951302 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.951280    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-workload-certs\") pod \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") "
Sep 09 22:09:28.951527 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.951377    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istiod-ca-cert\") pod \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") "
Sep 09 22:09:28.951527 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.951416    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-podinfo\") pod \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") "
Sep 09 22:09:28.951527 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.951458    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-credential-socket\") pod \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") "
Sep 09 22:09:28.951527 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.951493    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-workload-socket\") pod \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") "
Sep 09 22:09:28.951765 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.951546    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-envoy\") pod \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") "
Sep 09 22:09:28.951765 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.951546    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c5ac6049-fc4d-420d-b1f4-cb5788d49023" (UID: "c5ac6049-fc4d-420d-b1f4-cb5788d49023"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:09:28.951765 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.951593    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-data\") pod \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") "
Sep 09 22:09:28.951765 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.951633    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-token\") pod \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\" (UID: \"c5ac6049-fc4d-420d-b1f4-cb5788d49023\") "
Sep 09 22:09:28.952003 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.951792    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c5ac6049-fc4d-420d-b1f4-cb5788d49023" (UID: "c5ac6049-fc4d-420d-b1f4-cb5788d49023"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:09:28.952003 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.951815    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c5ac6049-fc4d-420d-b1f4-cb5788d49023" (UID: "c5ac6049-fc4d-420d-b1f4-cb5788d49023"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 22:09:28.952003 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.951823    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-data" (OuterVolumeSpecName: "istio-data") pod "c5ac6049-fc4d-420d-b1f4-cb5788d49023" (UID: "c5ac6049-fc4d-420d-b1f4-cb5788d49023"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:09:28.952184 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.952022    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-workload-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:28.952184 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.952027    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c5ac6049-fc4d-420d-b1f4-cb5788d49023" (UID: "c5ac6049-fc4d-420d-b1f4-cb5788d49023"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:09:28.952184 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.952043    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istiod-ca-cert\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:28.952184 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.952057    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-credential-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:28.952184 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.952070    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-data\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:28.953806 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.953771    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5ac6049-fc4d-420d-b1f4-cb5788d49023-kube-api-access-bn9jw" (OuterVolumeSpecName: "kube-api-access-bn9jw") pod "c5ac6049-fc4d-420d-b1f4-cb5788d49023" (UID: "c5ac6049-fc4d-420d-b1f4-cb5788d49023"). InnerVolumeSpecName "kube-api-access-bn9jw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:09:28.953923 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.953805    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c5ac6049-fc4d-420d-b1f4-cb5788d49023" (UID: "c5ac6049-fc4d-420d-b1f4-cb5788d49023"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 22:09:28.953923 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.953844    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c5ac6049-fc4d-420d-b1f4-cb5788d49023" (UID: "c5ac6049-fc4d-420d-b1f4-cb5788d49023"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 22:09:28.954147 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:28.954125    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-token" (OuterVolumeSpecName: "istio-token") pod "c5ac6049-fc4d-420d-b1f4-cb5788d49023" (UID: "c5ac6049-fc4d-420d-b1f4-cb5788d49023"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:09:29.053181 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.053149    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-workload-socket\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:29.053181 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.053176    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-envoy\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:29.053181 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.053186    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-token\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:29.053368 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.053194    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn9jw\" (UniqueName: \"kubernetes.io/projected/c5ac6049-fc4d-420d-b1f4-cb5788d49023-kube-api-access-bn9jw\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:29.053368 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.053204    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5ac6049-fc4d-420d-b1f4-cb5788d49023-istio-podinfo\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:09:29.229124 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.229035    2578 generic.go:356] "Generic (PLEG): container finished" podID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" containerID="6e16bdcb5f057677bb55ead46c65a3b0197746917be1e79fac3a6f956bdc783f" exitCode=0
Sep 09 22:09:29.229124 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.229086    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2" event={"ID":"c5ac6049-fc4d-420d-b1f4-cb5788d49023","Type":"ContainerDied","Data":"6e16bdcb5f057677bb55ead46c65a3b0197746917be1e79fac3a6f956bdc783f"}
Sep 09 22:09:29.229308 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.229138    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2" event={"ID":"c5ac6049-fc4d-420d-b1f4-cb5788d49023","Type":"ContainerDied","Data":"a72c215ad3885efb2a1e54a0de710cabe62e24547eb949a5187ff7b4b6db0ea0"}
Sep 09 22:09:29.229308 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.229166    2578 scope.go:117] "RemoveContainer" containerID="6e16bdcb5f057677bb55ead46c65a3b0197746917be1e79fac3a6f956bdc783f"
Sep 09 22:09:29.229308 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.229091    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"
Sep 09 22:09:29.238896 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.238875    2578 scope.go:117] "RemoveContainer" containerID="c864142a0916eed295f79e8bb6a155f3dd65d95f4290da4490b2ac39668755fb"
Sep 09 22:09:29.248030 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.248000    2578 scope.go:117] "RemoveContainer" containerID="6e16bdcb5f057677bb55ead46c65a3b0197746917be1e79fac3a6f956bdc783f"
Sep 09 22:09:29.248361 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:09:29.248333    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e16bdcb5f057677bb55ead46c65a3b0197746917be1e79fac3a6f956bdc783f\": container with ID starting with 6e16bdcb5f057677bb55ead46c65a3b0197746917be1e79fac3a6f956bdc783f not found: ID does not exist" containerID="6e16bdcb5f057677bb55ead46c65a3b0197746917be1e79fac3a6f956bdc783f"
Sep 09 22:09:29.248457 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.248372    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e16bdcb5f057677bb55ead46c65a3b0197746917be1e79fac3a6f956bdc783f"} err="failed to get container status \"6e16bdcb5f057677bb55ead46c65a3b0197746917be1e79fac3a6f956bdc783f\": rpc error: code = NotFound desc = could not find container \"6e16bdcb5f057677bb55ead46c65a3b0197746917be1e79fac3a6f956bdc783f\": container with ID starting with 6e16bdcb5f057677bb55ead46c65a3b0197746917be1e79fac3a6f956bdc783f not found: ID does not exist"
Sep 09 22:09:29.248457 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.248392    2578 scope.go:117] "RemoveContainer" containerID="c864142a0916eed295f79e8bb6a155f3dd65d95f4290da4490b2ac39668755fb"
Sep 09 22:09:29.248583 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.248550    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"]
Sep 09 22:09:29.248682 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:09:29.248664    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c864142a0916eed295f79e8bb6a155f3dd65d95f4290da4490b2ac39668755fb\": container with ID starting with c864142a0916eed295f79e8bb6a155f3dd65d95f4290da4490b2ac39668755fb not found: ID does not exist" containerID="c864142a0916eed295f79e8bb6a155f3dd65d95f4290da4490b2ac39668755fb"
Sep 09 22:09:29.248750 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.248685    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c864142a0916eed295f79e8bb6a155f3dd65d95f4290da4490b2ac39668755fb"} err="failed to get container status \"c864142a0916eed295f79e8bb6a155f3dd65d95f4290da4490b2ac39668755fb\": rpc error: code = NotFound desc = could not find container \"c864142a0916eed295f79e8bb6a155f3dd65d95f4290da4490b2ac39668755fb\": container with ID starting with c864142a0916eed295f79e8bb6a155f3dd65d95f4290da4490b2ac39668755fb not found: ID does not exist"
Sep 09 22:09:29.254904 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:29.254871    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-64fec6-openshift-default-798455f9fd-fl9b2"]
Sep 09 22:09:31.008444 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:09:31.004443    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" path="/var/lib/kubelet/pods/c5ac6049-fc4d-420d-b1f4-cb5788d49023/volumes"
Sep 09 22:10:48.981332 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:10:48.981298    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:11:15.766590 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:11:15.766553    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:11:23.568882 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:11:23.568847    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:11:31.774982 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:11:31.774936    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:11:49.775426 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:11:49.775394    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:11:56.676589 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:11:56.676512    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:13:01.236685 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:13:01.236654    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 22:13:01.244178 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:13:01.244154    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 22:13:23.580869 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:13:23.580830    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:13:28.581293 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:13:28.581261    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:13:36.571764 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:13:36.571722    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:13:44.381683 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:13:44.381641    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:14:34.265958 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:34.265922    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:14:41.775862 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:41.775821    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:14:45.976518 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:45.976487    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:14:46.186626 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.186591    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"]
Sep 09 22:14:46.186959 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.186947    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3184537a-9bb6-4dff-ae62-6e22dec826c3" containerName="revoke-keys"
Sep 09 22:14:46.187039 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.186961    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3184537a-9bb6-4dff-ae62-6e22dec826c3" containerName="revoke-keys"
Sep 09 22:14:46.187039 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.186993    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" containerName="istio-proxy"
Sep 09 22:14:46.187039 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.187003    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" containerName="istio-proxy"
Sep 09 22:14:46.187039 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.187021    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d131295-a2b3-4629-a34e-172b8e8bb828" containerName="istio-proxy"
Sep 09 22:14:46.187039 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.187029    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d131295-a2b3-4629-a34e-172b8e8bb828" containerName="istio-proxy"
Sep 09 22:14:46.187039 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.187038    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" containerName="istio-proxy"
Sep 09 22:14:46.187228 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.187044    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" containerName="istio-proxy"
Sep 09 22:14:46.187228 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.187057    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d131295-a2b3-4629-a34e-172b8e8bb828" containerName="istio-proxy"
Sep 09 22:14:46.187228 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.187063    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d131295-a2b3-4629-a34e-172b8e8bb828" containerName="istio-proxy"
Sep 09 22:14:46.187228 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.187124    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" containerName="istio-proxy"
Sep 09 22:14:46.187228 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.187131    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3184537a-9bb6-4dff-ae62-6e22dec826c3" containerName="revoke-keys"
Sep 09 22:14:46.187228 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.187140    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d131295-a2b3-4629-a34e-172b8e8bb828" containerName="istio-proxy"
Sep 09 22:14:46.187228 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.187147    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d131295-a2b3-4629-a34e-172b8e8bb828" containerName="istio-proxy"
Sep 09 22:14:46.190102 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.190085    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:46.193658 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.193633    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 09 22:14:46.193795 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.193674    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 22:14:46.193795 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.193701    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 22:14:46.193795 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.193717    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qxw5s\""
Sep 09 22:14:46.193795 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.193680    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 22:14:46.200145 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.200119    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"]
Sep 09 22:14:46.300054 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.300023    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93d2923e-3160-4731-88b5-76199b9f30e2-maas-api-tls\") pod \"maas-api-55c59f7b5b-7dlbz\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") " pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:46.300205 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.300057    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93d2923e-3160-4731-88b5-76199b9f30e2-openshift-service-ca\") pod \"maas-api-55c59f7b5b-7dlbz\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") " pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:46.300205 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.300098    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqn6g\" (UniqueName: \"kubernetes.io/projected/93d2923e-3160-4731-88b5-76199b9f30e2-kube-api-access-xqn6g\") pod \"maas-api-55c59f7b5b-7dlbz\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") " pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:46.300205 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.300113    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93d2923e-3160-4731-88b5-76199b9f30e2-metrics-certs\") pod \"maas-api-55c59f7b5b-7dlbz\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") " pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:46.400835 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.400807    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93d2923e-3160-4731-88b5-76199b9f30e2-maas-api-tls\") pod \"maas-api-55c59f7b5b-7dlbz\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") " pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:46.400944 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.400837    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93d2923e-3160-4731-88b5-76199b9f30e2-openshift-service-ca\") pod \"maas-api-55c59f7b5b-7dlbz\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") " pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:46.400944 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.400879    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqn6g\" (UniqueName: \"kubernetes.io/projected/93d2923e-3160-4731-88b5-76199b9f30e2-kube-api-access-xqn6g\") pod \"maas-api-55c59f7b5b-7dlbz\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") " pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:46.400944 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.400904    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93d2923e-3160-4731-88b5-76199b9f30e2-metrics-certs\") pod \"maas-api-55c59f7b5b-7dlbz\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") " pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:46.401588 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.401566    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93d2923e-3160-4731-88b5-76199b9f30e2-openshift-service-ca\") pod \"maas-api-55c59f7b5b-7dlbz\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") " pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:46.403398 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.403377    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93d2923e-3160-4731-88b5-76199b9f30e2-maas-api-tls\") pod \"maas-api-55c59f7b5b-7dlbz\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") " pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:46.403496 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.403440    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93d2923e-3160-4731-88b5-76199b9f30e2-metrics-certs\") pod \"maas-api-55c59f7b5b-7dlbz\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") " pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:46.408257 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.408238    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqn6g\" (UniqueName: \"kubernetes.io/projected/93d2923e-3160-4731-88b5-76199b9f30e2-kube-api-access-xqn6g\") pod \"maas-api-55c59f7b5b-7dlbz\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") " pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:46.502140 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.502116    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:46.776155 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.776130    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"]
Sep 09 22:14:46.777760 ip-10-0-140-157 kubenswrapper[2578]: W0909 22:14:46.777720    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93d2923e_3160_4731_88b5_76199b9f30e2.slice/crio-6054493d4d970f6dbf977cf60394558f63d1d3229d31f14433e60e83239b05eb WatchSource:0}: Error finding container 6054493d4d970f6dbf977cf60394558f63d1d3229d31f14433e60e83239b05eb: Status 404 returned error can't find the container with id 6054493d4d970f6dbf977cf60394558f63d1d3229d31f14433e60e83239b05eb
Sep 09 22:14:46.779201 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.779181    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 22:14:46.981919 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:46.981850    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz" event={"ID":"93d2923e-3160-4731-88b5-76199b9f30e2","Type":"ContainerStarted","Data":"6054493d4d970f6dbf977cf60394558f63d1d3229d31f14433e60e83239b05eb"}
Sep 09 22:14:47.990194 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:47.990156    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz" event={"ID":"93d2923e-3160-4731-88b5-76199b9f30e2","Type":"ContainerStarted","Data":"87178199f5d88c49a5b79c830fea99379f05221a1f9b9156b7071fd649bd37a2"}
Sep 09 22:14:47.990580 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:47.990223    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:48.008448 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:48.008406    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz" podStartSLOduration=1.621887512 podStartE2EDuration="2.008391513s" totalImagesPullingTime="386.504001ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:14:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:14:48.008148682 +0000 UTC m=+2507.481113592" watchObservedRunningTime="2026-09-09 22:14:48.008391513 +0000 UTC m=+2507.481356423"
Sep 09 22:14:53.868882 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:53.868805    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:14:54.001213 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.001187    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:14:54.649184 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.649153    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"]
Sep 09 22:14:54.649611 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.649597    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5ac6049-fc4d-420d-b1f4-cb5788d49023" containerName="istio-proxy"
Sep 09 22:14:54.652813 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.652796    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:14:54.662267 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.662245    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"]
Sep 09 22:14:54.772841 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.772809    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df6cfda2-7376-4155-9963-48c3b96a1fce-openshift-service-ca\") pod \"maas-api-6c5bd889d8-pkcch\" (UID: \"df6cfda2-7376-4155-9963-48c3b96a1fce\") " pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:14:54.773007 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.772858    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df6cfda2-7376-4155-9963-48c3b96a1fce-metrics-certs\") pod \"maas-api-6c5bd889d8-pkcch\" (UID: \"df6cfda2-7376-4155-9963-48c3b96a1fce\") " pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:14:54.773007 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.772885    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df6cfda2-7376-4155-9963-48c3b96a1fce-maas-api-tls\") pod \"maas-api-6c5bd889d8-pkcch\" (UID: \"df6cfda2-7376-4155-9963-48c3b96a1fce\") " pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:14:54.773007 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.772909    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfnl5\" (UniqueName: \"kubernetes.io/projected/df6cfda2-7376-4155-9963-48c3b96a1fce-kube-api-access-nfnl5\") pod \"maas-api-6c5bd889d8-pkcch\" (UID: \"df6cfda2-7376-4155-9963-48c3b96a1fce\") " pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:14:54.874319 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.874278    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df6cfda2-7376-4155-9963-48c3b96a1fce-openshift-service-ca\") pod \"maas-api-6c5bd889d8-pkcch\" (UID: \"df6cfda2-7376-4155-9963-48c3b96a1fce\") " pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:14:54.874727 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.874357    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df6cfda2-7376-4155-9963-48c3b96a1fce-metrics-certs\") pod \"maas-api-6c5bd889d8-pkcch\" (UID: \"df6cfda2-7376-4155-9963-48c3b96a1fce\") " pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:14:54.874727 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.874408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df6cfda2-7376-4155-9963-48c3b96a1fce-maas-api-tls\") pod \"maas-api-6c5bd889d8-pkcch\" (UID: \"df6cfda2-7376-4155-9963-48c3b96a1fce\") " pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:14:54.874727 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.874435    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfnl5\" (UniqueName: \"kubernetes.io/projected/df6cfda2-7376-4155-9963-48c3b96a1fce-kube-api-access-nfnl5\") pod \"maas-api-6c5bd889d8-pkcch\" (UID: \"df6cfda2-7376-4155-9963-48c3b96a1fce\") " pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:14:54.875290 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.875264    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df6cfda2-7376-4155-9963-48c3b96a1fce-openshift-service-ca\") pod \"maas-api-6c5bd889d8-pkcch\" (UID: \"df6cfda2-7376-4155-9963-48c3b96a1fce\") " pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:14:54.877299 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.877272    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df6cfda2-7376-4155-9963-48c3b96a1fce-maas-api-tls\") pod \"maas-api-6c5bd889d8-pkcch\" (UID: \"df6cfda2-7376-4155-9963-48c3b96a1fce\") " pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:14:54.877516 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.877494    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df6cfda2-7376-4155-9963-48c3b96a1fce-metrics-certs\") pod \"maas-api-6c5bd889d8-pkcch\" (UID: \"df6cfda2-7376-4155-9963-48c3b96a1fce\") " pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:14:54.885025 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.884999    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfnl5\" (UniqueName: \"kubernetes.io/projected/df6cfda2-7376-4155-9963-48c3b96a1fce-kube-api-access-nfnl5\") pod \"maas-api-6c5bd889d8-pkcch\" (UID: \"df6cfda2-7376-4155-9963-48c3b96a1fce\") " pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:14:54.966408 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:54.966341    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:14:55.250811 ip-10-0-140-157 kubenswrapper[2578]: W0909 22:14:55.250782    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf6cfda2_7376_4155_9963_48c3b96a1fce.slice/crio-59cd1f5606f5be7109f646e86751100b39727c8b3ea4b201704f15522bb9c568 WatchSource:0}: Error finding container 59cd1f5606f5be7109f646e86751100b39727c8b3ea4b201704f15522bb9c568: Status 404 returned error can't find the container with id 59cd1f5606f5be7109f646e86751100b39727c8b3ea4b201704f15522bb9c568
Sep 09 22:14:55.251562 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:55.251466    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"]
Sep 09 22:14:56.044491 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:56.044458    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch" event={"ID":"df6cfda2-7376-4155-9963-48c3b96a1fce","Type":"ContainerStarted","Data":"52c87860d219433912c16335c0c2bedc683c54d6f84cc9112efe7c2ee04081b9"}
Sep 09 22:14:56.044873 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:56.044509    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch" event={"ID":"df6cfda2-7376-4155-9963-48c3b96a1fce","Type":"ContainerStarted","Data":"59cd1f5606f5be7109f646e86751100b39727c8b3ea4b201704f15522bb9c568"}
Sep 09 22:14:56.044873 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:56.044581    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:14:56.063700 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:14:56.063654    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch" podStartSLOduration=1.7087361859999999 podStartE2EDuration="2.063640701s" totalImagesPullingTime="354.904515ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:14:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:14:56.06225268 +0000 UTC m=+2515.535217601" watchObservedRunningTime="2026-09-09 22:14:56.063640701 +0000 UTC m=+2515.536605680"
Sep 09 22:15:00.870750 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:00.870714    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:15:02.055442 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:02.055414    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6c5bd889d8-pkcch"
Sep 09 22:15:02.101714 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:02.101680    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"]
Sep 09 22:15:02.101910 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:02.101881    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz" podUID="93d2923e-3160-4731-88b5-76199b9f30e2" containerName="maas-api" containerID="cri-o://87178199f5d88c49a5b79c830fea99379f05221a1f9b9156b7071fd649bd37a2" gracePeriod=30
Sep 09 22:15:03.346298 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:03.346275    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:15:03.451218 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:03.451148    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqn6g\" (UniqueName: \"kubernetes.io/projected/93d2923e-3160-4731-88b5-76199b9f30e2-kube-api-access-xqn6g\") pod \"93d2923e-3160-4731-88b5-76199b9f30e2\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") "
Sep 09 22:15:03.451218 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:03.451194    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93d2923e-3160-4731-88b5-76199b9f30e2-metrics-certs\") pod \"93d2923e-3160-4731-88b5-76199b9f30e2\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") "
Sep 09 22:15:03.451412 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:03.451280    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93d2923e-3160-4731-88b5-76199b9f30e2-maas-api-tls\") pod \"93d2923e-3160-4731-88b5-76199b9f30e2\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") "
Sep 09 22:15:03.451412 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:03.451324    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93d2923e-3160-4731-88b5-76199b9f30e2-openshift-service-ca\") pod \"93d2923e-3160-4731-88b5-76199b9f30e2\" (UID: \"93d2923e-3160-4731-88b5-76199b9f30e2\") "
Sep 09 22:15:03.451729 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:03.451700    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93d2923e-3160-4731-88b5-76199b9f30e2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "93d2923e-3160-4731-88b5-76199b9f30e2" (UID: "93d2923e-3160-4731-88b5-76199b9f30e2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 22:15:03.453303 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:03.453280    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93d2923e-3160-4731-88b5-76199b9f30e2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "93d2923e-3160-4731-88b5-76199b9f30e2" (UID: "93d2923e-3160-4731-88b5-76199b9f30e2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 22:15:03.453419 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:03.453343    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93d2923e-3160-4731-88b5-76199b9f30e2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "93d2923e-3160-4731-88b5-76199b9f30e2" (UID: "93d2923e-3160-4731-88b5-76199b9f30e2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 22:15:03.453466 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:03.453425    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93d2923e-3160-4731-88b5-76199b9f30e2-kube-api-access-xqn6g" (OuterVolumeSpecName: "kube-api-access-xqn6g") pod "93d2923e-3160-4731-88b5-76199b9f30e2" (UID: "93d2923e-3160-4731-88b5-76199b9f30e2"). InnerVolumeSpecName "kube-api-access-xqn6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 22:15:03.551991 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:03.551947    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93d2923e-3160-4731-88b5-76199b9f30e2-openshift-service-ca\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:15:03.552131 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:03.551994    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqn6g\" (UniqueName: \"kubernetes.io/projected/93d2923e-3160-4731-88b5-76199b9f30e2-kube-api-access-xqn6g\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:15:03.552131 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:03.552011    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93d2923e-3160-4731-88b5-76199b9f30e2-metrics-certs\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:15:03.552131 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:03.552025    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93d2923e-3160-4731-88b5-76199b9f30e2-maas-api-tls\") on node \"ip-10-0-140-157.ec2.internal\" DevicePath \"\""
Sep 09 22:15:04.091843 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:04.091809    2578 generic.go:356] "Generic (PLEG): container finished" podID="93d2923e-3160-4731-88b5-76199b9f30e2" containerID="87178199f5d88c49a5b79c830fea99379f05221a1f9b9156b7071fd649bd37a2" exitCode=0
Sep 09 22:15:04.092043 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:04.091863    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"
Sep 09 22:15:04.092043 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:04.091893    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz" event={"ID":"93d2923e-3160-4731-88b5-76199b9f30e2","Type":"ContainerDied","Data":"87178199f5d88c49a5b79c830fea99379f05221a1f9b9156b7071fd649bd37a2"}
Sep 09 22:15:04.092043 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:04.091942    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz" event={"ID":"93d2923e-3160-4731-88b5-76199b9f30e2","Type":"ContainerDied","Data":"6054493d4d970f6dbf977cf60394558f63d1d3229d31f14433e60e83239b05eb"}
Sep 09 22:15:04.092043 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:04.091981    2578 scope.go:117] "RemoveContainer" containerID="87178199f5d88c49a5b79c830fea99379f05221a1f9b9156b7071fd649bd37a2"
Sep 09 22:15:04.101306 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:04.101287    2578 scope.go:117] "RemoveContainer" containerID="87178199f5d88c49a5b79c830fea99379f05221a1f9b9156b7071fd649bd37a2"
Sep 09 22:15:04.101562 ip-10-0-140-157 kubenswrapper[2578]: E0909 22:15:04.101540    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87178199f5d88c49a5b79c830fea99379f05221a1f9b9156b7071fd649bd37a2\": container with ID starting with 87178199f5d88c49a5b79c830fea99379f05221a1f9b9156b7071fd649bd37a2 not found: ID does not exist" containerID="87178199f5d88c49a5b79c830fea99379f05221a1f9b9156b7071fd649bd37a2"
Sep 09 22:15:04.101639 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:04.101573    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87178199f5d88c49a5b79c830fea99379f05221a1f9b9156b7071fd649bd37a2"} err="failed to get container status \"87178199f5d88c49a5b79c830fea99379f05221a1f9b9156b7071fd649bd37a2\": rpc error: code = NotFound desc = could not find container \"87178199f5d88c49a5b79c830fea99379f05221a1f9b9156b7071fd649bd37a2\": container with ID starting with 87178199f5d88c49a5b79c830fea99379f05221a1f9b9156b7071fd649bd37a2 not found: ID does not exist"
Sep 09 22:15:04.113357 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:04.113335    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"]
Sep 09 22:15:04.117416 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:04.117395    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-55c59f7b5b-7dlbz"]
Sep 09 22:15:05.002351 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:05.002313    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93d2923e-3160-4731-88b5-76199b9f30e2" path="/var/lib/kubelet/pods/93d2923e-3160-4731-88b5-76199b9f30e2/volumes"
Sep 09 22:15:07.168941 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:07.168912    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:15:13.972561 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:13.972525    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkzhw"]
Sep 09 22:15:22.062444 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:22.062408    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7l92p_14b3dc4c-298c-49cd-abe5-bbd1c54a87fb/manager/0.log"
Sep 09 22:15:22.179674 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:22.179645    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7tdjr_95e3f673-2fac-4251-82e0-433956f63519/manager/0.log"
Sep 09 22:15:22.432825 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:22.432757    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-88srq_533f307f-ef83-4756-9186-2aa44b9eacc6/server/0.log"
Sep 09 22:15:22.562710 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:22.562683    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-v8xm7_05cf0125-a374-4a44-97ba-e9fdd79eee32/manager/0.log"
Sep 09 22:15:23.061254 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:23.061223    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84fbbd4d59-wh2vj_03067d81-ba5a-44ca-abe3-6c5f9f1e166c/copy-manifests/0.log"
Sep 09 22:15:23.073569 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:23.073544    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84fbbd4d59-wh2vj_03067d81-ba5a-44ca-abe3-6c5f9f1e166c/manager/0.log"
Sep 09 22:15:23.695389 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:23.695364    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6c5bd889d8-pkcch_df6cfda2-7376-4155-9963-48c3b96a1fce/maas-api/0.log"
Sep 09 22:15:23.811280 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:23.811253    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29816505-l9dxb_20cb7fae-d483-4337-b7fc-dffb21361761/cleanup/0.log"
Sep 09 22:15:23.924318 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:23.924297    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29816520-bxtb2_80370a40-f00b-45e0-846e-a80189ee0cc4/cleanup/0.log"
Sep 09 22:15:25.306531 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:25.306480    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-fl4hs_2b9d853d-c684-4128-8bbd-5ab7f3fdbc8f/manager/0.log"
Sep 09 22:15:25.419346 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:25.419322    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-j7w47_074ab1c6-f610-4bc1-9cca-56eefee785e8/kuadrant-console-plugin/0.log"
Sep 09 22:15:25.710807 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:25.710746    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-lkzhw_e7157711-a94f-4a96-a2e5-3875233ebf2c/limitador/0.log"
Sep 09 22:15:26.287878 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:26.287846    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls_d3e588e7-7b28-45e8-a99b-a7b2467a5916/istio-proxy/0.log"
Sep 09 22:15:26.801749 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:26.801716    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-vxvgt_e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab/istio-proxy/1.log"
Sep 09 22:15:27.179336 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:27.179260    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7685746588-wcx7f_ab94f25b-423a-42a4-8784-2004122e692d/router/0.log"
Sep 09 22:15:27.733651 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:27.733620    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-ldnp8_8615776e-9ce8-4608-90c4-8fa84c8a6341/main/0.log"
Sep 09 22:15:34.770255 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:34.770220    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j2gqd/must-gather-6bx2r"]
Sep 09 22:15:34.770649 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:34.770592    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93d2923e-3160-4731-88b5-76199b9f30e2" containerName="maas-api"
Sep 09 22:15:34.770649 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:34.770604    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="93d2923e-3160-4731-88b5-76199b9f30e2" containerName="maas-api"
Sep 09 22:15:34.770728 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:34.770669    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="93d2923e-3160-4731-88b5-76199b9f30e2" containerName="maas-api"
Sep 09 22:15:34.774195 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:34.774174    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j2gqd/must-gather-6bx2r"
Sep 09 22:15:34.777761 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:34.777726    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j2gqd\"/\"kube-root-ca.crt\""
Sep 09 22:15:34.778509 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:34.778487    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-j2gqd\"/\"default-dockercfg-7b6pt\""
Sep 09 22:15:34.778652 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:34.778524    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j2gqd\"/\"openshift-service-ca.crt\""
Sep 09 22:15:34.783007 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:34.782983    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j2gqd/must-gather-6bx2r"]
Sep 09 22:15:34.888339 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:34.888302    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0318c30b-8dfb-45c6-bf30-74533435a11e-must-gather-output\") pod \"must-gather-6bx2r\" (UID: \"0318c30b-8dfb-45c6-bf30-74533435a11e\") " pod="openshift-must-gather-j2gqd/must-gather-6bx2r"
Sep 09 22:15:34.888468 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:34.888350    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr2s4\" (UniqueName: \"kubernetes.io/projected/0318c30b-8dfb-45c6-bf30-74533435a11e-kube-api-access-wr2s4\") pod \"must-gather-6bx2r\" (UID: \"0318c30b-8dfb-45c6-bf30-74533435a11e\") " pod="openshift-must-gather-j2gqd/must-gather-6bx2r"
Sep 09 22:15:34.989309 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:34.989270    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0318c30b-8dfb-45c6-bf30-74533435a11e-must-gather-output\") pod \"must-gather-6bx2r\" (UID: \"0318c30b-8dfb-45c6-bf30-74533435a11e\") " pod="openshift-must-gather-j2gqd/must-gather-6bx2r"
Sep 09 22:15:34.989309 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:34.989318    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr2s4\" (UniqueName: \"kubernetes.io/projected/0318c30b-8dfb-45c6-bf30-74533435a11e-kube-api-access-wr2s4\") pod \"must-gather-6bx2r\" (UID: \"0318c30b-8dfb-45c6-bf30-74533435a11e\") " pod="openshift-must-gather-j2gqd/must-gather-6bx2r"
Sep 09 22:15:34.989653 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:34.989631    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0318c30b-8dfb-45c6-bf30-74533435a11e-must-gather-output\") pod \"must-gather-6bx2r\" (UID: \"0318c30b-8dfb-45c6-bf30-74533435a11e\") " pod="openshift-must-gather-j2gqd/must-gather-6bx2r"
Sep 09 22:15:34.997709 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:34.997683    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr2s4\" (UniqueName: \"kubernetes.io/projected/0318c30b-8dfb-45c6-bf30-74533435a11e-kube-api-access-wr2s4\") pod \"must-gather-6bx2r\" (UID: \"0318c30b-8dfb-45c6-bf30-74533435a11e\") " pod="openshift-must-gather-j2gqd/must-gather-6bx2r"
Sep 09 22:15:35.085009 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:35.084939    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j2gqd/must-gather-6bx2r"
Sep 09 22:15:35.354298 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:35.354271    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j2gqd/must-gather-6bx2r"]
Sep 09 22:15:35.355860 ip-10-0-140-157 kubenswrapper[2578]: W0909 22:15:35.355834    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0318c30b_8dfb_45c6_bf30_74533435a11e.slice/crio-8351beef08ab07791073a7157db90567ed904eca432e89a173904cafd0f667d1 WatchSource:0}: Error finding container 8351beef08ab07791073a7157db90567ed904eca432e89a173904cafd0f667d1: Status 404 returned error can't find the container with id 8351beef08ab07791073a7157db90567ed904eca432e89a173904cafd0f667d1
Sep 09 22:15:36.272788 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:36.272748    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j2gqd/must-gather-6bx2r" event={"ID":"0318c30b-8dfb-45c6-bf30-74533435a11e","Type":"ContainerStarted","Data":"8351beef08ab07791073a7157db90567ed904eca432e89a173904cafd0f667d1"}
Sep 09 22:15:37.285044 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:37.285009    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j2gqd/must-gather-6bx2r" event={"ID":"0318c30b-8dfb-45c6-bf30-74533435a11e","Type":"ContainerStarted","Data":"1191e89728bace12ade2263a16a57dd7f8ccd126337768b5e48e293d049d499a"}
Sep 09 22:15:37.285558 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:37.285053    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j2gqd/must-gather-6bx2r" event={"ID":"0318c30b-8dfb-45c6-bf30-74533435a11e","Type":"ContainerStarted","Data":"046c56a3127b17585fb21d031a780d222189d59fdb9848b189944a5d864ca0b3"}
Sep 09 22:15:37.301118 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:37.301065    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-j2gqd/must-gather-6bx2r" podStartSLOduration=2.343134176 podStartE2EDuration="3.30104519s" totalImagesPullingTime="957.911014ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 22:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:15:37.300658308 +0000 UTC m=+2556.773623219" watchObservedRunningTime="2026-09-09 22:15:37.30104519 +0000 UTC m=+2556.774010100"
Sep 09 22:15:37.946557 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:37.946526    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-69ls5_1995b548-a1df-4e1f-81ea-0f66bc72bf58/global-pull-secret-syncer/0.log"
Sep 09 22:15:38.122702 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:38.122667    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m7d8c_c297d41d-7ddc-4eaa-9982-77b1b8dbb645/konnectivity-agent/0.log"
Sep 09 22:15:38.222669 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:38.222563    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-157.ec2.internal_58cbb1667332061624da09e390b4d934/haproxy/0.log"
Sep 09 22:15:42.566724 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:42.566696    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-fl4hs_2b9d853d-c684-4128-8bbd-5ab7f3fdbc8f/manager/0.log"
Sep 09 22:15:42.612862 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:42.612835    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-j7w47_074ab1c6-f610-4bc1-9cca-56eefee785e8/kuadrant-console-plugin/0.log"
Sep 09 22:15:42.689378 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:42.689349    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-lkzhw_e7157711-a94f-4a96-a2e5-3875233ebf2c/limitador/0.log"
Sep 09 22:15:44.572404 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:44.572360    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7f785858c4-m5bd7_cb4d3ab4-c14d-4698-909e-1cd3bc466e58/metrics-server/0.log"
Sep 09 22:15:44.598716 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:44.598685    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-2588p_9b3d4319-b5e0-4f29-a0e4-fa4669144864/monitoring-plugin/0.log"
Sep 09 22:15:44.699510 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:44.699482    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jkn9v_35e853f0-ec73-457e-9a6d-a5c4af34588f/node-exporter/0.log"
Sep 09 22:15:44.728768 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:44.728745    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jkn9v_35e853f0-ec73-457e-9a6d-a5c4af34588f/kube-rbac-proxy/0.log"
Sep 09 22:15:44.744559 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:44.744532    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jkn9v_35e853f0-ec73-457e-9a6d-a5c4af34588f/init-textfile/0.log"
Sep 09 22:15:44.864195 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:44.864108    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-v2gwl_f0a6de4a-b76b-4eb2-9e9b-b1452d8881be/kube-rbac-proxy-main/0.log"
Sep 09 22:15:44.890411 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:44.890339    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-v2gwl_f0a6de4a-b76b-4eb2-9e9b-b1452d8881be/kube-rbac-proxy-self/0.log"
Sep 09 22:15:44.913550 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:44.913498    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-v2gwl_f0a6de4a-b76b-4eb2-9e9b-b1452d8881be/openshift-state-metrics/0.log"
Sep 09 22:15:45.133918 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:45.133840    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-ccczb_fef07e24-eca1-47da-b1ae-5adb2bc040ac/prometheus-operator/0.log"
Sep 09 22:15:45.159235 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:45.159201    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-ccczb_fef07e24-eca1-47da-b1ae-5adb2bc040ac/kube-rbac-proxy/0.log"
Sep 09 22:15:45.194356 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:45.194325    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-fzxp6_19be090a-b269-4250-9702-a60e5e6eaa20/prometheus-operator-admission-webhook/0.log"
Sep 09 22:15:46.922749 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:46.922723    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-mb2cr_ee4873ec-62e9-4389-bc83-7b2736c20d08/networking-console-plugin/0.log"
Sep 09 22:15:47.594203 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:47.594176    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/1.log"
Sep 09 22:15:47.600703 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:47.600673    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-mksnd_14f44884-6479-48c1-9961-717810d01fc8/console-operator/2.log"
Sep 09 22:15:48.334221 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.334184    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"]
Sep 09 22:15:48.342694 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.342665    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.348252 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.348220    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"]
Sep 09 22:15:48.433930 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.433883    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e76438fd-906e-495b-bbc9-b7d93ef02f68-lib-modules\") pod \"perf-node-gather-daemonset-5lncl\" (UID: \"e76438fd-906e-495b-bbc9-b7d93ef02f68\") " pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.434144 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.434042    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvsdq\" (UniqueName: \"kubernetes.io/projected/e76438fd-906e-495b-bbc9-b7d93ef02f68-kube-api-access-hvsdq\") pod \"perf-node-gather-daemonset-5lncl\" (UID: \"e76438fd-906e-495b-bbc9-b7d93ef02f68\") " pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.434144 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.434086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e76438fd-906e-495b-bbc9-b7d93ef02f68-podres\") pod \"perf-node-gather-daemonset-5lncl\" (UID: \"e76438fd-906e-495b-bbc9-b7d93ef02f68\") " pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.434144 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.434120    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e76438fd-906e-495b-bbc9-b7d93ef02f68-sys\") pod \"perf-node-gather-daemonset-5lncl\" (UID: \"e76438fd-906e-495b-bbc9-b7d93ef02f68\") " pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.434305 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.434230    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e76438fd-906e-495b-bbc9-b7d93ef02f68-proc\") pod \"perf-node-gather-daemonset-5lncl\" (UID: \"e76438fd-906e-495b-bbc9-b7d93ef02f68\") " pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.535453 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.535418    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvsdq\" (UniqueName: \"kubernetes.io/projected/e76438fd-906e-495b-bbc9-b7d93ef02f68-kube-api-access-hvsdq\") pod \"perf-node-gather-daemonset-5lncl\" (UID: \"e76438fd-906e-495b-bbc9-b7d93ef02f68\") " pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.535453 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.535457    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e76438fd-906e-495b-bbc9-b7d93ef02f68-podres\") pod \"perf-node-gather-daemonset-5lncl\" (UID: \"e76438fd-906e-495b-bbc9-b7d93ef02f68\") " pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.535707 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.535491    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e76438fd-906e-495b-bbc9-b7d93ef02f68-sys\") pod \"perf-node-gather-daemonset-5lncl\" (UID: \"e76438fd-906e-495b-bbc9-b7d93ef02f68\") " pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.535707 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.535559    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e76438fd-906e-495b-bbc9-b7d93ef02f68-proc\") pod \"perf-node-gather-daemonset-5lncl\" (UID: \"e76438fd-906e-495b-bbc9-b7d93ef02f68\") " pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.535707 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.535618    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e76438fd-906e-495b-bbc9-b7d93ef02f68-lib-modules\") pod \"perf-node-gather-daemonset-5lncl\" (UID: \"e76438fd-906e-495b-bbc9-b7d93ef02f68\") " pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.535707 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.535618    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e76438fd-906e-495b-bbc9-b7d93ef02f68-sys\") pod \"perf-node-gather-daemonset-5lncl\" (UID: \"e76438fd-906e-495b-bbc9-b7d93ef02f68\") " pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.535707 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.535640    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e76438fd-906e-495b-bbc9-b7d93ef02f68-podres\") pod \"perf-node-gather-daemonset-5lncl\" (UID: \"e76438fd-906e-495b-bbc9-b7d93ef02f68\") " pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.536012 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.535701    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e76438fd-906e-495b-bbc9-b7d93ef02f68-proc\") pod \"perf-node-gather-daemonset-5lncl\" (UID: \"e76438fd-906e-495b-bbc9-b7d93ef02f68\") " pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.536012 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.535739    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e76438fd-906e-495b-bbc9-b7d93ef02f68-lib-modules\") pod \"perf-node-gather-daemonset-5lncl\" (UID: \"e76438fd-906e-495b-bbc9-b7d93ef02f68\") " pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.543644 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.543620    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvsdq\" (UniqueName: \"kubernetes.io/projected/e76438fd-906e-495b-bbc9-b7d93ef02f68-kube-api-access-hvsdq\") pod \"perf-node-gather-daemonset-5lncl\" (UID: \"e76438fd-906e-495b-bbc9-b7d93ef02f68\") " pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.664741 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.664662    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:48.910340 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.910305    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-rhkfk_2f5dc97e-22f3-457d-bc7e-ad1e262691ba/volume-data-source-validator/0.log"
Sep 09 22:15:48.975238 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:48.974805    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"]
Sep 09 22:15:48.976727 ip-10-0-140-157 kubenswrapper[2578]: W0909 22:15:48.976692    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode76438fd_906e_495b_bbc9_b7d93ef02f68.slice/crio-0ff0475b7f786124182bea8a72436bcdc038d54a6c2337dbff96f0abf9f1ac15 WatchSource:0}: Error finding container 0ff0475b7f786124182bea8a72436bcdc038d54a6c2337dbff96f0abf9f1ac15: Status 404 returned error can't find the container with id 0ff0475b7f786124182bea8a72436bcdc038d54a6c2337dbff96f0abf9f1ac15
Sep 09 22:15:49.401081 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:49.401036    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl" event={"ID":"e76438fd-906e-495b-bbc9-b7d93ef02f68","Type":"ContainerStarted","Data":"686e3d0a4c108fe2028289446848a43f2394e7c13833fa5773ca7ad432856bf0"}
Sep 09 22:15:49.401805 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:49.401100    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl" event={"ID":"e76438fd-906e-495b-bbc9-b7d93ef02f68","Type":"ContainerStarted","Data":"0ff0475b7f786124182bea8a72436bcdc038d54a6c2337dbff96f0abf9f1ac15"}
Sep 09 22:15:49.402343 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:49.402319    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:49.420237 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:49.420186    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl" podStartSLOduration=1.420167575 podStartE2EDuration="1.420167575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 22:15:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 22:15:49.417051753 +0000 UTC m=+2568.890016669" watchObservedRunningTime="2026-09-09 22:15:49.420167575 +0000 UTC m=+2568.893132487"
Sep 09 22:15:50.153233 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:50.153192    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lcmzv_849ad12b-744d-4efa-ace8-0ba4dec756f1/dns/0.log"
Sep 09 22:15:50.172804 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:50.172780    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lcmzv_849ad12b-744d-4efa-ace8-0ba4dec756f1/kube-rbac-proxy/0.log"
Sep 09 22:15:50.236875 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:50.236848    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jrklh_1a022781-ea0f-4915-96be-242aac7202e6/dns-node-resolver/0.log"
Sep 09 22:15:50.863552 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:50.863520    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wcng5_a97e012d-3878-4dc9-a0c4-9f0dae4cd093/node-ca/0.log"
Sep 09 22:15:51.761924 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:51.761893    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls_d3e588e7-7b28-45e8-a99b-a7b2467a5916/istio-proxy/0.log"
Sep 09 22:15:52.426746 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:52.426714    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-vxvgt_e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab/istio-proxy/0.log"
Sep 09 22:15:52.430141 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:52.430109    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-vxvgt_e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab/istio-proxy/1.log"
Sep 09 22:15:52.525687 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:52.525661    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7685746588-wcx7f_ab94f25b-423a-42a4-8784-2004122e692d/router/0.log"
Sep 09 22:15:53.393730 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:53.393706    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-59n6x_21d909c2-ce09-479f-b353-8b23ce49954d/serve-healthcheck-canary/0.log"
Sep 09 22:15:54.039257 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:54.039217    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mwjgp_e282a8be-3ba4-42d0-ade2-241c1cf8a144/kube-rbac-proxy/0.log"
Sep 09 22:15:54.060467 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:54.060434    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mwjgp_e282a8be-3ba4-42d0-ade2-241c1cf8a144/exporter/0.log"
Sep 09 22:15:54.081191 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:54.081161    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mwjgp_e282a8be-3ba4-42d0-ade2-241c1cf8a144/extractor/0.log"
Sep 09 22:15:56.199727 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:56.199675    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7l92p_14b3dc4c-298c-49cd-abe5-bbd1c54a87fb/manager/0.log"
Sep 09 22:15:56.424878 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:56.424852    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-j2gqd/perf-node-gather-daemonset-5lncl"
Sep 09 22:15:56.591263 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:56.591233    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7tdjr_95e3f673-2fac-4251-82e0-433956f63519/manager/0.log"
Sep 09 22:15:56.691934 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:56.691906    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-88srq_533f307f-ef83-4756-9186-2aa44b9eacc6/server/0.log"
Sep 09 22:15:56.798484 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:56.798448    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-v8xm7_05cf0125-a374-4a44-97ba-e9fdd79eee32/manager/0.log"
Sep 09 22:15:56.929424 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:56.929296    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84fbbd4d59-wh2vj_03067d81-ba5a-44ca-abe3-6c5f9f1e166c/manager/0.log"
Sep 09 22:15:56.946092 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:56.946066    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84fbbd4d59-wh2vj_03067d81-ba5a-44ca-abe3-6c5f9f1e166c/copy-manifests/0.log"
Sep 09 22:15:58.342907 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:58.342872    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-64d94cf4d-hjk2v_3ff621b4-7fe0-496a-a072-b41fbd411abc/manager/0.log"
Sep 09 22:15:58.373709 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:58.373685    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls_d3e588e7-7b28-45e8-a99b-a7b2467a5916/istio-proxy/0.log"
Sep 09 22:15:58.949260 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:58.949219    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-vxvgt_e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab/istio-proxy/0.log"
Sep 09 22:15:58.953868 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:58.953840    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-vxvgt_e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab/istio-proxy/1.log"
Sep 09 22:15:59.030740 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:15:59.030713    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7685746588-wcx7f_ab94f25b-423a-42a4-8784-2004122e692d/router/0.log"
Sep 09 22:16:00.582745 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:00.582713    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cxv4ls_d3e588e7-7b28-45e8-a99b-a7b2467a5916/istio-proxy/0.log"
Sep 09 22:16:01.220782 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:01.220747    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-vxvgt_e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab/istio-proxy/0.log"
Sep 09 22:16:01.224672 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:01.224636    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-vxvgt_e3be0db2-ed98-4a15-ad0f-7f1b3e3833ab/istio-proxy/1.log"
Sep 09 22:16:01.298955 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:01.298924    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7685746588-wcx7f_ab94f25b-423a-42a4-8784-2004122e692d/router/0.log"
Sep 09 22:16:01.605833 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:01.605757    2578 scope.go:117] "RemoveContainer" containerID="6f245e66ed7712e75a50b34b5f7c6bf9004a949290e2834f3bd3af4c4567865f"
Sep 09 22:16:03.924317 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:03.924236    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-6nvrn_226b4308-cb75-45b3-aaa6-467856e4d5eb/kube-storage-version-migrator-operator/1.log"
Sep 09 22:16:03.925239 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:03.925218    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-6nvrn_226b4308-cb75-45b3-aaa6-467856e4d5eb/kube-storage-version-migrator-operator/0.log"
Sep 09 22:16:04.859828 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:04.859797    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4zq5s_cc8427f7-1d50-457a-879b-001799103caa/kube-multus/0.log"
Sep 09 22:16:05.027844 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:05.027814    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rqzxb_414900e1-c678-4e61-9e76-0e2e29ac783c/kube-multus-additional-cni-plugins/0.log"
Sep 09 22:16:05.047381 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:05.047355    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rqzxb_414900e1-c678-4e61-9e76-0e2e29ac783c/egress-router-binary-copy/0.log"
Sep 09 22:16:05.066748 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:05.066723    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rqzxb_414900e1-c678-4e61-9e76-0e2e29ac783c/cni-plugins/0.log"
Sep 09 22:16:05.086057 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:05.086032    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rqzxb_414900e1-c678-4e61-9e76-0e2e29ac783c/bond-cni-plugin/0.log"
Sep 09 22:16:05.105219 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:05.105200    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rqzxb_414900e1-c678-4e61-9e76-0e2e29ac783c/routeoverride-cni/0.log"
Sep 09 22:16:05.125312 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:05.125260    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rqzxb_414900e1-c678-4e61-9e76-0e2e29ac783c/whereabouts-cni-bincopy/0.log"
Sep 09 22:16:05.144504 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:05.144487    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rqzxb_414900e1-c678-4e61-9e76-0e2e29ac783c/whereabouts-cni/0.log"
Sep 09 22:16:05.519621 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:05.519587    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sgvmj_7c340b83-fda4-4fb8-afdd-9ec6d597ec62/network-metrics-daemon/0.log"
Sep 09 22:16:05.539467 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:05.539443    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sgvmj_7c340b83-fda4-4fb8-afdd-9ec6d597ec62/kube-rbac-proxy/0.log"
Sep 09 22:16:06.569650 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:06.569625    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rftp9_d7a07ed1-b2fb-48ef-b337-677f0aa1a385/ovn-controller/0.log"
Sep 09 22:16:06.588124 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:06.588100    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rftp9_d7a07ed1-b2fb-48ef-b337-677f0aa1a385/ovn-acl-logging/0.log"
Sep 09 22:16:06.618979 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:06.618943    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rftp9_d7a07ed1-b2fb-48ef-b337-677f0aa1a385/kube-rbac-proxy-node/0.log"
Sep 09 22:16:06.648818 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:06.648793    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rftp9_d7a07ed1-b2fb-48ef-b337-677f0aa1a385/kube-rbac-proxy-ovn-metrics/0.log"
Sep 09 22:16:06.672084 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:06.672063    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rftp9_d7a07ed1-b2fb-48ef-b337-677f0aa1a385/northd/0.log"
Sep 09 22:16:06.694459 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:06.694438    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rftp9_d7a07ed1-b2fb-48ef-b337-677f0aa1a385/nbdb/0.log"
Sep 09 22:16:06.715982 ip-10-0-140-157 kubenswrapper[2578]: I0909 22:16:06.715944    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rftp9_d7a07ed1-b2fb-48ef-b337-677f0aa1a385/sbdb/0.log"